Weird missing file...

[stamatis]

Hi all,
not sure if this is something to do with internal housekeeping with LiveCloud...

I've added a small custom library to the /CanelaDB/libraries/ folder to manage the initialising code no matter the location of this directory, which works well for my purposes. 

Only, on occasion the stack file simply disappears!? (ie the above directory will suddenly only contain the 2 canelaDB libraries). 
Not offloaded to cloud - just gone.
There is absolutely nothing in my stack that would delete any file and I've not encountered this behaviour on Mac at any point in the last 30+ years...
In fact i now keep a copy of this little lib in the parent directory (all of these are in a writeable location) to keep copying it back to /CanelaDB/libraries/. This extra file never disappears, so it's unlikely to be a Mac thing.

So i have to ask - is this some internal housekeeping thing by liveCloud? Should i not be keeping other files in this directory?

Many thanks
Stam

-- edit: i should note this happens in the IDE only, i've not tested with standalones; IDE is 9.6.5 (stable) and OS is MacOS Monterey 12.0.1. LiveCloud libraries are the most up-to-date at this point in time.

[clarencemartin]

Hi all,
not sure if this is something to do with internal housekeeping with LiveCloud...

I've added a small custom library to the /CanelaDB/libraries/ folder to manage the initialising code no matter the location of this directory, which works well for my purposes. 

Only, on occasion the stack file simply disappears!? (ie the above directory will suddenly only contain the 2 canelaDB libraries). 
Not offloaded to cloud - just gone.
There is absolutely nothing in my stack that would delete any file and I've not encountered this behaviour on Mac at any point in the last 30+ years...
In fact i now keep a copy of this little lib in the parent directory (all of these are in a writeable location) to keep copying it back to /CanelaDB/libraries/. This extra file never disappears, so it's unlikely to be a Mac thing.

So i have to ask - is this some internal housekeeping thing by liveCloud? Should i not be keeping other files in this directory?

Many thanks
Stam

-- edit: i should note this happens in the IDE only, i've not tested with standalones; IDE is 9.6.5 (stable) and OS is MacOS Monterey 12.0.1. LiveCloud libraries are the most up-to-date at this point in time.

---

Hi Stamatis,
I suggest that you create a separate folder to store your custom library in.
I do a similar procedure in my applications and I load them on a preOpen or Open command.
I think that when you do a Canela Liveclouddb update, that is when you lose your library.

---

this is a copy of the script that I use to load my libraries:
Code:
command Load_App_Libraries, tAppLibraryContents
  local tPath, tStack, tLibraryNames, tLContents, tLNames
  --THE FOLLOWING CODE WILL only LOAD LIBRARY FILES
  filter tAppLibraryContents without empty
  filter tAppLibraryContents with "*.lib"
  repeat for each line xStackName in tAppLibraryContents
     try
        start using xStackName
     catch error
        answer "Please check your Library stack" & cr & " There is an error in your stack " & xStackName
        exit to top
     end try
     
  end repeat
 
end Load_App_Libraries

[clarencemartin]

Sorry, I forgot some code that is necessary.

this is the complete code:

command initializeAppLibraries
local tAppLibraryPath, tLibraryContents
global LOFlag

set the itemDel to "/"

if the environment is "development" then
put item 1 to -2 of (the effective fileName of this stack) into tAppLibraryPath
else
switch the platform
case "android"
case "iPhone"
put specialFolderPath("engine") into tAppLibraryPath
break

case "MacOS"
put specialFolderPath("resources") into tAppLibraryPath
break

case "Win32"
case "Linux"
put item 1 to -2 of (the effective fileName of this stack) into tAppLibraryPath
break
end switch
end if

set the defaultfolder to tAppLibraryPath
put "/App_libraries/" after tAppLibraryPath
set the defaultfolder to tAppLibraryPath
put the files after tLibraryContents
-- if LOFlag is true then
-- unLoad_App_Libraries tLibraryContents, tAppLibraryPath
-- end if
Load_App_Libraries tLibraryContents, tAppLibraryPath
end initializeAppLibraries

command Load_App_Libraries, tAppLibraryContents
local tPath, tStack, tLibraryNames, tLContents, tLNames
--THE FOLLOWING CODE WILL only LOAD LIBRARY FILES
filter tAppLibraryContents without empty
filter tAppLibraryContents with "*.lib"
repeat for each line xStackName in tAppLibraryContents
try
start using xStackName
catch error
answer "Please check your Library stack" & cr & " There is an error in your stack " & xStackName
exit to top
end try

end repeat

end Load_App_Libraries

[stamatis]

Thanks Clarence - i'm well aware of the issues with upgrading the libraries (the first 10 times i fell for that gradually taught me the lesson lol)
I always download the libraries to a neutral location and copy them manually to where i want them to be so as to not overwrite existing folders that contain my files.

What i'm talking about is just randomly (no upgrade) my app stops working with the error that it can't find my custom library and it just seems to do a vanishing act. This has now happened several times, which is getting a bit weird.


The reason i've set it up this way is that i have a number of project folders and instead of replicating code in each app (and increasing risk of tiresome errors), i do this via my little custom library which resides inside CanelaDB and therefore always knows the relative path to the lib files and just loads them with 2 lines of code and keeps my working stack free of clutter.
And likewise with library upgrades, i just have the one central file to update with new checksums etc (well, 1 per project anyway)

This works very well for me - except for the mysterious vanishing act of the file. Initially i thought maybe it was something to do with time machine, but that doesn't make sense and copies of the same file in the same root directory are always unaffected. This happens literally between sessions of closing and opening the mainstack.


On further reflection i wonder if this may have something to do with my closeStack code. In this, i want to unload all stack files from memory then close the mainstack. I used the 'delete stack <stackName>' command as according to the documentation this removes stacks from memory and shouldn't actually delete them, but i wonder...

[mark_talluto]

Hi Stam,

To be clear, are you putting extra files inside your CanelaDB/libraries folder?

There was a time where we would delete files that did not belong (not from Canela). I need to review the start-up process to see if that is still there. For the time being, it would be best to use another folder for non-CanelaDB files.

CanelaDB will not look outside of its purview in your folder structure.

I'll report back what I find.

[mark_talluto]

The libraries look to be safe. They do not appear to delete files for any reason in the library folder of CanelaDB. They do prune files as needed in CanelaDB/database/ folder and CanelaDB/config/ folders.

On the other hand, LCM does aggressively clean out files inside of its CanelaDB/libraries/ folder. I will paste the code below.

Code:

command initializeCanelaDB
   local tLibraryPath

   wait 0 milliseconds with messages ## GIVE THE ENGINE A CHANCE TO SPIN UP
   put "LCM-AUTH-CODE-HERE" into gCDBAuthKey
   set the itemDel to "/"

   if the environment is "development" then
        put item 1 to -2 of (the effective fileName of this stack) into tLibraryPath
   else
        switch the platform
             case "android"
             case "iPhone"
                  put specialFolderPath("engine") into tLibraryPath
                  break

             case "MacOS"
                  put specialFolderPath("support") into tLibraryPath
                  put "/" & "CanelaSoftware/" & gCSManifestA["productName"] & "/" & gCSManifestA["productVersion"] & "/data" after tLibraryPath --LCM SPECIFIC PATH
                  break

             case "Win32"
             case "Linux"
                  put item 1 to -2 of (the effective fileName of this stack) into tLibraryPath
                  break
        end switch
   end if

   put "/config/CanelaDB/libraries/" after tLibraryPath

   if the environment <> "development" then
        repeat for each line xLine in files(tLibraryPath)
             if xLine <> "CDB_Header.lib" and xLine <> "CDB_Starter.lib" then
                  delete file (tLibraryPath & xLine)
             end if
        end repeat
   else
        if there is a file (tLibraryPath & ".DS_Store") then
             delete file (tLibraryPath & ".DS_Store")
        end if
   end if

   --START CanelaDB
   csi_verifyLibraries tLibraryPath

   try
        start using stack (tLibraryPath & "CDB_Header.lib")
        start using stack (tLibraryPath & "CDB_Starter.lib")

   catch tError
        answer "There was an error initializing CanelaDB:" && tError
        exit to top
   end try

   --SET OPTIONAL LOGGING MODE
   cdb_SetLogMode "message box"

   cdb_loadTable "cdbCache"
end initializeCanelaDB

[stamatis]

On the other hand, LCM does aggressively clean out files inside of its CanelaDB/libraries/ folder. I will paste the code below.

Thanks Mark,
my custom lib lives in /CanelaDB/Libraries/ - presumably this should not be affected by LCM?
This was just convenience/minimising code to have single global variable pointing to both the liveCloud libs and my helper lib.

I'll just change this...

[mark_talluto]

You are correct. LCM should not reach in there. I am curious to hear if your problem goes away once you have a separate location for your libraries. If it shows up again, or not, we will have learned something.

[clarencemartin]

Mark,
I just performed the following test:
First off, I copied a library into my CanelaDB /libraries folder.
I then did an update of my LCM for that project.
The update deleted the library test file from the CanelaDb/libraries folder.

[mark_talluto]

Ok. That jives with LCM being responsible for the libraries folder. I think we figured no one would put things in there. If everyone feels strongly about using that folder, we could update LCM to not own that folder the way it does today. We would delete only the two canelaDB library files and replace them with new versions. All other files could be left alone.

Please post your thoughts.