View Issue Details

IDProjectCategoryView StatusLast Update
0008758ADK-Xojo-V4RBRBDB APIpublic2020-04-09 13:44
ReporterDeltaworX Assigned ToRuslan Zasukhin  
PrioritynormalSeveritymajorReproducibilityalways
Status resolvedResolutionfixed 
PlatformINTELOSMAC OS OS Version*
Product Version10.0.x 
Fixed in Version10.1.x 
Summary0008758: RBDB API - no data returned from BLOB field
DescriptionA PNG Picture is stored into a BLOB field in VRBDatabase (local or remote on VServer).

Trying to get the correctly stored data back in a rowset results in number 0 or 1 as content. Even base64 encoded data is returned as number 0 or 1.

Plugin Version 10.1.1
Steps To ReproduceCreate a table in Xojo 2019r3.1
DB.ExecuteSQL("CREATE TABLE IF NOT EXISTS test (ID INTEGER NOT NULL PRIMARY KEY, logo BLOB);")

Store data into an existing record in that table (e.g. LogoPic.Backdrop is a canvas with a dropped picture as backdrop)
DB.ExecuteSQL("UPDATE test SET logo = ? WHERE ID = ?", LogoPic.Backdrop.GetData(Picture.FormatPNG), 0), 1 )

Try to retrieve the stored data
Var rs As RowSet
rs = DB.SelectSQL("SELECT * FROM test WHERE ID = ?", 1)

Var p As Picture
p = rs.Column("logo").PictureValue //.NativeValue also does not work...
TagsNo tags attached.

Activities

Ruslan Zasukhin

Ruslan Zasukhin

2020-03-30 12:34

administrator   ~0011181

Hi,

thank you for bug report.
We will try check this issue asap.

meanwhile, why you try to use RbDB API?
Valentina API is much more rich and effective.
Any strong reason for such a choice?
Ruslan Zasukhin

Ruslan Zasukhin

2020-03-30 13:56

administrator   ~0011182

for info: in 2019r2 they deprecate DatabaseField, and new is: DatabaseColumn

https://docs.xojo.com/DatabaseField .PictureValue .NativeValue
https://docs.xojo.com/DatabaseColumn. removed .NativeValue

Not clear for now how from Variant a picture comes ...
 https://docs.xojo.com/Variant

I do not see here type Picture at all, and do not see any Binary value..

OLD .PictureValue was able to control the format of pictures and compression.
How this is done in a Variant? Before value is stored in it?
Ruslan Zasukhin

Ruslan Zasukhin

2020-03-30 13:58

administrator   ~0011183

aha, they have made method PictureValue() instead of Property .PictureValue

https://docs.xojo.com/DatabaseColumn.PictureValue

although in docs they say about OLD DatabaseField.PictureField:
  Please use DatabaseColumn.Value as a replacement.
DeltaworX

DeltaworX

2020-03-30 15:53

reporter   ~0011184

Problem is not only with Pictures stored in a BLOB field, but also with String stored in BLOB field. it is not possible to get any data out of the BLOB field, regardless if I use .PictureValue, .NativeValue, .StringValue or .Value
Ruslan Zasukhin

Ruslan Zasukhin

2020-03-30 15:54

administrator   ~0011185

yes, in RBDB was not implemented Binary fields...

So far I have study mySQL sources.

Interesting is that on old Cursor class - they also did not have support of BLOBs in the ColumnValue...
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-03 10:18

administrator   ~0011191

I think I have got the first success.

todo more tests and more ways ...
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-05 12:31

administrator   ~0011193

Last edited: 2020-04-05 12:32

8758: V4RB: FIX RBDB API for Valentina DB: cursor does not return BLOB - Picture.
* This was not implemented originally.
* Now added into switch support for BLOB Picture, also FixedBinary/VarBinary.
 * For BLOB and Picture, we need to create buffers where read BLOB value from db.
 * For this RZ have added MemPool into cursor class, also two vectors mBuffers and mSizes.
 * Also ClearBuffers() which is called on any change of the current record.

* New tests added into TestProject of V4RB for RBDB_Valentina_BLOBs
* Btw, we did not have any RBDB tests.

POTENTIAL PROBLEM:
our FixedBinary/VarBinary returns values via REALstring, which seems to be able keep BIANRY values
we return length and buffer. Xojo correctly shows that in debugger.
But when DO_TEST_EQ() did add of these strings to Console.textarea, RAM of Xojo grow to gigabytes and they show garbage after a string.
IMHO Xojo has a bug at least in this place, they do not check length but search zero-end.

TODO:
* I have noticed that Valentina API classes when working with parameter REALarray inBinds, just check it for nil,
while RBDB classes yet do a check of UpperBound() and comment point that Xojo can give us NOT NULL REALobject for an empty array.
I think we need to do here some polish - providing a SINGLE method for all these places!!

DeltaworX

DeltaworX

2020-04-08 09:01

reporter   ~0011199

Thanks for your support. Did just test version 10.1.2 - storing into BLOB fields works (as far as I can see), but reading from BLOB field results in cut off files. I do store Base64 encoded text and images in BLOB field and when reading, data is is truncated - in textfile parts of the text is missing, same in jpegs.

Anything to be aware of when reading data from the blob fields?

Storing:
data is defined a String and contains Base64 encoded filedata...
DB.ExecuteSQL("INSERT INTO files (file) VALUES (?)", data )

Reading:
DB.SelectSQL("SELECT file FROM files WHERE ID = ?", id)
file is then Base64 decoded and saved to disk
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-08 12:05

administrator   ~0011200

Hi,

1) well STORING I did not touch in this update, so it works as it was.

2) READING was absent at all, now it is added for BLOB, Picture, and FixedBinary / VarBinary types of Valentina DB

3) You can look into my test in Examples / TestProjects / RBDB_Valentina / Test_BLOBs

But I have test here picture and binary fields, BLOB field was not touched in test.
Picture did work for me. -- JPG stored in our VPicture field

Code for Picture and BLOB in our c++ code of V4RB 100% same ..
Because of this, so far I wonder where and how truncation could happen...


Well, can you give me a simple project (1-2 functions) which reproduces your steps?
I will use it to debug issue.
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-08 12:32

administrator   ~0011201

DB.ExecuteSQL("INSERT INTO files (file) VALUES (?)", data )

data is array of strings, right ?
DeltaworX

DeltaworX

2020-04-08 12:36

reporter   ~0011202

Looks like problem comes from storing data in BLOB field. I have attached a small TEST project...
TEST.xojo_binary_project.zip (7,335 bytes)
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-08 12:51

administrator   ~0011203

thank you for project!
I will play with it asap.
Ruslan Zasukhin

Ruslan Zasukhin

2020-04-08 14:29

administrator   ~0011204

Okay, look please.

* So in db.ExecuteSql( query, variant1, variant2 ). -- we give array of VARIANT values.
* You give String containing binary data to be stored into BLOB field.

But in our code we do right now this (in case you get c++)

            case 8: // String
            case 37: // Text
            {
                REALstring sv;
                res = REALGetPropValueString( inRealObject, "StringValue", &sv );

                Str_Unicode suv( sv );
                long length = suv.length();

                a = CreateValueString( fNullable, &length, NULL );

                // sv == nullptr (and length == 0) is a Xojo's empty string, and it must be not
                // an SQL-NULL bind value. If you need to bind a NULL you should use real Xoho's
                // NULL.
                if( length )
                    a->put_String( suv.c_str(), suv.end() );
                else
                    a->put_String( "" );
            }break;

            case 9: // Object
            {
                REALmemoryBlock sv = (REALmemoryBlock)inRealObject;
                vuint32 length = REALMemoryBlockGetSize(sv);
                char* pRawData = (char*)REALMemoryBlockGetPtr(sv);

                a = CreateValueFixedBinary( fNullable, &length, NULL );
                a->put_String( pRawData, pRawData + length );

            }break;


* Anyway, main point is that if we see String, we try to convert its encoding. What changes the size of data x2

* And for BINARY data we expect to get. VARIANT value with MemoryBlock.
  Can you try to send your data using MemoryBlock?
  I think in this way it should work fine.
DeltaworX

DeltaworX

2020-04-09 07:01

reporter   ~0011205

Yes. working with memoryblock solves the problem of truncated stored data. Thanks!

Issue History

Date Modified Username Field Change
2020-03-27 11:42 DeltaworX New Issue
2020-03-30 12:34 Ruslan Zasukhin Note Added: 0011181
2020-03-30 13:56 Ruslan Zasukhin Note Added: 0011182
2020-03-30 13:58 Ruslan Zasukhin Note Added: 0011183
2020-03-30 15:53 DeltaworX Note Added: 0011184
2020-03-30 15:54 Ruslan Zasukhin Note Added: 0011185
2020-04-03 10:18 Ruslan Zasukhin Note Added: 0011191
2020-04-05 12:31 Ruslan Zasukhin Note Added: 0011193
2020-04-05 12:32 Ruslan Zasukhin Note Edited: 0011193
2020-04-05 12:39 Ruslan Zasukhin Assigned To => Ruslan Zasukhin
2020-04-05 12:39 Ruslan Zasukhin Status new => resolved
2020-04-05 12:39 Ruslan Zasukhin Resolution open => fixed
2020-04-05 12:39 Ruslan Zasukhin Fixed in Version => 10.1.x
2020-04-08 09:01 DeltaworX Status resolved => feedback
2020-04-08 09:01 DeltaworX Resolution fixed => reopened
2020-04-08 09:01 DeltaworX Note Added: 0011199
2020-04-08 12:05 Ruslan Zasukhin Note Added: 0011200
2020-04-08 12:32 Ruslan Zasukhin Note Added: 0011201
2020-04-08 12:36 DeltaworX File Added: TEST.xojo_binary_project.zip
2020-04-08 12:36 DeltaworX Note Added: 0011202
2020-04-08 12:36 DeltaworX Status feedback => assigned
2020-04-08 12:51 Ruslan Zasukhin Note Added: 0011203
2020-04-08 14:29 Ruslan Zasukhin Note Added: 0011204
2020-04-09 07:01 DeltaworX Note Added: 0011205
2020-04-09 13:44 Ruslan Zasukhin Status assigned => resolved
2020-04-09 13:44 Ruslan Zasukhin Resolution reopened => fixed