import CSV file size limit?

[sltfn]

Hello,

I have a lot of records to import. Curious what the recommended file size limit is, if any, for importing CSV files via the LiveCloud Manager? I don't want to bite off more than it can chew at one time.

Separate but related issue, I am currently importing data using a LiveCode script but seem to run into problems if I try too many records at once; maybe I need to make my script more efficient. Thanks in advance.

Sieg

[jasonchan]

The only size limitation is how much data a variable can hold in memory in LiveCode because the importing in LiveCloud Manager loads your file in memory and parse it.
However, I am not sure what that size is. Maybe others with more experience in LiveCode can tell you.

For your own script, what problem do you run into? Also what platform are you on? Currently, Windows is slower on arrayencoding large data than Mac which can be the problem you run into. Other than that, there shouldn't be a problem, it might just take some time to upload a large amount of data.

[sltfn]

The only size limitation is how much data a variable can hold in memory in LiveCode because the importing in LiveCloud Manager loads your file in memory and parse it.
However, I am not sure what that size is. Maybe others with more experience in LiveCode can tell you.

For your own script, what problem do you run into? Also what platform are you on? Currently, Windows is slower on arrayencoding large data than Mac which can be the problem you run into. Other than that, there shouldn't be a problem, it might just take some time to upload a large amount of data.

Thanks, Jason. I'm on Mac.

A large part of my present learning curve is that I'm new to arrays, conceptually and practically. I've pasted some tab-delimited sample data bearing no relation to what I actually need to upload below. In fact, the table I'm uploading has 22 keys, not just 4.

ID    CUSTOMERS    CATEGORY    STATUS
1    Joe Smith    RETAIL    1
2    Carol Jones    WHOLESALE    2
3    Tom Johnson    RETAIL    2
4    Joan Williams    RETAIL    3
(MANY records follow)

As far as I can tell, there is no way in LiveCode to split this array by column AND row so that the top line entries (ID, CUSTOMERS, CATEGORY, STATUS) are recognized as keys in a 2-dimensional array that includes all the other entries. The only option I've tried successfully, and it does work, is to loop through my tab-delimited list line by line, split each line by tab, which yields a 1-dimensional array with keys [1], [2], [3], [4]. Then loop through each of the elements in that array and put them into an array with keys [ID], [CUSTOMERS], [CATEGORY] and [STATUS] that match my table in LiveCloud, then upload that second array, representing a single line of my data, to LiveCloud. Is there an alternate method in LiveCode to encode the whole list as a 2-dimensional array with a single split command and upload it?

Alternately is there a twist on the LiveCloud cdb_create call that would, if my array has numbered keys [1], [2], [3], [4] allow me to put the data attached to each of those keys into the correct fields in my LiveCloud table, 1 into ID, 2 into CUSTOMERS, 3 into CATEGORY and 4 into STATUS?

Perhaps I'm not explaining this well but could use any tips you may have relative to array encoding data efficiently in LiveCode so that the cdb_create call puts the columnar data in each line of my list into the correct LiveCloud field in my table. Thanks in advance.

[jasonchan]

I think you got everything already. There is not really a good way to do it currently that I know off.
Another thing you can try is to have it as a JSON file and call JsonToArray.
Luckily, this conversion should only happen one time and you shouldn't need to translate any more after that,