LiveCloud standalone issues...

[stamatis]

Hi Mark and team
Some quick questions on a liveCloud-enabled standalone:

I had only included mergJSON with initial builds (as per the website). This seemed to fail, and resolved when i included the standard JSON library as well. Are both required or is this a coincidence?

To ensure connectivity with liveCloud include the ‘canelaDB’ folder as an in the standalone settings ‘copy files’ section (ie should be available in specialFolderPath("resources"). On first run, this folder is copied to the user’s Documents folder (writeable location), along with he mainstack that is launched form the splash stack.
However when building a mac app, the locked libraries are causing the standalone build to fail with the error ‘cannot open output files’ or some such. I can work around this by removing the the CanelaDB folder from the ‘Copy Files’ section of the standalone settings, build the app and then manually copy this to <show app package contents>/Contents/Resources/_MacOS/
Is there a better way to do this?

On running a the same standalone on Mac and Windows, it works fine on Mac, but on Windows it crashes - after some head scratching it seems that the checksum library checks on Mac work fine but throw an error on Windows. I’ve worked around this by commenting out the checksum library checks - the app works beutifully and will be used in a very trusted environment so that’s fine, but it’s obviously not desirable otherwise.
Can multiple checksums be correct (ie should i have checksum that works for both mac and win, unlike the one i have now that only works for mac)? Or do i need to derive a Windows checksum and include a platform check in the library checks?

Many thanks as always
Stam

[mark_talluto]

I had only included mergJSON with initial builds (as per the website). This seemed to fail, and resolved when i included the standard JSON library as well. Are both required or is this a coincidence?

CanelaDB does not need the standard JSON library to be included when making your standalone. I verified that LCM does not have it included. 

To ensure connectivity with liveCloud include the ‘canelaDB’ folder as an in the standalone settings ‘copy files’ section (ie should be available in specialFolderPath("resources"). On first run, this folder is copied to the user’s Documents folder (writeable location), along with he mainstack that is launched form the splash stack.
However when building a mac app, the locked libraries are causing the standalone build to fail with the error ‘cannot open output files’ or some such. I can work around this by removing the the CanelaDB folder from the ‘Copy Files’ section of the standalone settings, build the app and then manually copy this to <show app package contents>/Contents/Resources/_MacOS/
Is there a better way to do this?

This sounds like a LiveCode standalone builder bug. I'll give it a try and report back what I find. By chance, are you using the "Search for required inclusions when saving the standalone application" when building? LiveCode will not be able to read through encrypted files to see what is inside. They should fail nicely and continue to work in my opinion. Maybe it is not doing that gracefully.

On running a the same standalone on Mac and Windows, it works fine on Mac, but on Windows it crashes - after some head scratching it seems that the checksum library checks on Mac work fine but throw an error on Windows. I’ve worked around this by commenting out the checksum library checks - the app works beutifully and will be used in a very trusted environment so that’s fine, but it’s obviously not desirable otherwise.
Can multiple checksums be correct (ie should i have checksum that works for both mac and win, unlike the one i have now that only works for mac)? Or do i need to derive a Windows checksum and include a platform check in the library checks?

The checksums should work for both platforms. After all, LiveCloud Manager relies on identical checksums for both platforms. It uses the same libraries we output when you do an export toolkit. Is it possible you are not using the same libraries for both platforms? Maybe an older version is being used on the Windows side?

-Mark

[stamatis]

I had only included mergJSON with initial builds (as per the website). This seemed to fail, and resolved when i included the standard JSON library as well. Are both required or is this a coincidence?

CanelaDB does not need the standard JSON library to be included when making your standalone. I verified that LCM does not have it included. 

To ensure connectivity with liveCloud include the ‘canelaDB’ folder as an in the standalone settings ‘copy files’ section (ie should be available in specialFolderPath("resources"). On first run, this folder is copied to the user’s Documents folder (writeable location), along with he mainstack that is launched form the splash stack.
However when building a mac app, the locked libraries are causing the standalone build to fail with the error ‘cannot open output files’ or some such. I can work around this by removing the the CanelaDB folder from the ‘Copy Files’ section of the standalone settings, build the app and then manually copy this to <show app package contents>/Contents/Resources/_MacOS/
Is there a better way to do this?

This sounds like a LiveCode standalone builder bug. I'll give it a try and report back what I find. By chance, are you using the "Search for required inclusions when saving the standalone application" when building? LiveCode will not be able to read through encrypted files to see what is inside. They should fail nicely and continue to work in my opinion. Maybe it is not doing that gracefully.

On running a the same standalone on Mac and Windows, it works fine on Mac, but on Windows it crashes - after some head scratching it seems that the checksum library checks on Mac work fine but throw an error on Windows. I’ve worked around this by commenting out the checksum library checks - the app works beutifully and will be used in a very trusted environment so that’s fine, but it’s obviously not desirable otherwise.
Can multiple checksums be correct (ie should i have checksum that works for both mac and win, unlike the one i have now that only works for mac)? Or do i need to derive a Windows checksum and include a platform check in the library checks?

The checksums should work for both platforms. After all, LiveCloud Manager relies on identical checksums for both platforms. It uses the same libraries we output when you do an export toolkit. Is it possible you are not using the same libraries for both platforms? Maybe an older version is being used on the Windows side?

-Mark

Hi Mark,
Re point 1: Noted, thank you. Probably coincidence then.

Re point 2: I absolutely do not let LC search for required inclusions automatically (really my comment on point 1 should have cleared that up ). 
All inclusions are selected manually and I  ensured that the 4 listed on the website are included (Internet, SSL, mergJSON and tsNet). 

The reason i even tried to add the locked files manually *after* the build is that i saw a post somewhere describing difficulties with locked files in this respect. 

It should be noted however that the Windows build is uneventful - it's just building for Big Sur that causes this problem for me. It's fine, just seems inelegant. But if you are able to build for Big Sur and including a folder containing the CanelaDB folder in the 'copy files' section please do let me know. (Mac OS 11.5, LC 9.6.2).

Re point 3: I absolutely do use the same library files, there is no question there. I should note however that the checksums i used were derived by me, as at the time the checksums in the stack script provided in the 'export toolkit' screen were incorrect and i wonder if i somehow managed to create a monstrosity. I presume by now the issue with the stack script checksums has been corrected, so will use these now and see if that resolves the issue.

---- EDIT -----
I did go back and look at the checksums again; On Mac these but both CDB_Starter.lib and CDB_Header.lib pass the checksum test just fine.
On Windows, with identical lib files the check on CDB_Header.lib throws the error "Error, CDB_Header checksum did not match."
The checksums i derived are now identical to ones provided in stack script in the 'export toolkit' screen. 
When i get a chance I'll download the lib files again and see if replacing them makes any difference...

Many thanks as always,
Stam

[mark_talluto]

Hi Mark,

The reason i even tried to add the locked files manually *after* the build is that i saw a post somewhere describing difficulties with locked files in this respect. 

I usually add my files and folders manually. I use this method out of a matter of preference more than anything else.

It should be noted however that the Windows build is uneventful - it's just building for Big Sur that causes this problem for me. It's fine, just seems inelegant. But if you are able to build for Big Sur and including a folder containing the CanelaDB folder in the 'copy files' section please do let me know. (Mac OS 11.5, LC 9.6.2).

I tried building both Mac and Win from Catalina. The first time I tried, I got an error. I redid the reference to the canelaDB folder in the standalone builder, and it worked for the next six tries. 

The apps ran on Mac and Windows without errors.

I tried the same thing on Big Sur. 
The standalone and the app running in the IDE ran without throwing any errors.

In all of my tests, I used the output CanelaDB folder that comes from LCM's ExportToolKit feature. I did not modify the initializeCanelaDB code in any way.

All of this testing has reminded me our easy it used to be to develop for Mac. It has gotten more complex in the name of security. I'll save that rant for another time.

---- EDIT -----
I did go back and look at the checksums again; On Mac these but both CDB_Starter.lib and CDB_Header.lib pass the checksum test just fine.
On Windows, with identical lib files the check on CDB_Header.lib throws the error "Error, CDB_Header checksum did not match."
The checksums i derived are now identical to ones provided in stack script in the 'export toolkit' screen. 
When i get a chance I'll download the lib files again and see if replacing them makes any difference...

If your LCM is outputting libraries and init code that does not jive, let us know. I did my tests with a shipping version of LCM. The issue could be that LCM is not updated to the current release. Would you mind sending me your LCM log file for review? You can find the log in the SpiceKit folder of your LCM installation.

[stamatis]

Hi Mark

In the end i think i narrowed the problem down to the folder i was trying to include in the 'copy files' section. This folder contains the CanelaDB folder and the mainstack that is launched from the splash stack; the idea is that these are copied to the user's Documents folder on first run as a writeable location.

I discovered if i removed this folder and *only* added the canelaDB folder the standalone would build as expected; and I added the mainstack to the 'stacks' screen (just needed an extra line of code to make sure these both end up in the same place in the user's Documents folder). I also had to disable messages while building to prevent the mainstack from being launched during the build...

With regards to the checksum issue, I'll see if i can replicate the issue again on the target Windows system; keep in mind that LCM will not run on that system (there is something in the firewall that is I think blocking some function of spiceKit, as per a previous support email - missing it's launcher). Will the log file from my mac be any help?

Many thanks for you help,
Stam

--- EDIT --- 
I also forgot to mention that adding the files manually seemed to work OK when running it off my system. But when uploading the file and then re-downloading, MacOS would completely stop this from running stating that the app had been tampered with (which it had  )  so i was very keen i could build this directly without having to manually add. Probably something that can be fixed when notarising etc, but easier not to have that issue!

[mark_talluto]

In the end i think i narrowed the problem down to the folder i was trying to include in the 'copy files' section. This folder contains the CanelaDB folder and the mainstack that is launched from the splash stack; the idea is that these are copied to the user's Documents folder on first run as a writeable location.

That is a good design. LCM and other apps we have made use of this model. Distributed apps for Mac will need to factor this in going forward. 

I discovered if i removed this folder and *only* added the canelaDB folder the standalone would build as expected; and I added the mainstack to the 'stacks' screen (just needed an extra line of code to make sure these both end up in the same place in the user's Documents folder). I also had to disable messages while building to prevent the mainstack from being launched during the build...

There was a change in the build process that now fires preOpenStack, openStack, and closeStack messages when using the standalone builder. 
If you are project is affected by this, you could add code like the following to help.

on closeStack
if the mode of stack "revStandaloneProgress" > 0 then
exit closesStack
end if
end closeStack

With regards to the checksum issue, I'll see if i can replicate the issue again on the target Windows system; keep in mind that LCM will not run on that system (there is something in the firewall that is I think blocking some function of spiceKit, as per a previous support email - missing it's launcher). Will the log file from my mac be any help?

For the time being, you can export the toolKit from the system that can run LCM. The target operating systems should be able to use the output from the building process just fine.

I would need the log to the system that is failing. It might give us some insight. If the firewall is blocking interaction with our servers, you can typically reverse any block from the Firewall UI of Windows. I had accidentally blocked apps before and had to go in there and adjust the inbound and outbound rules. 

--- EDIT --- 
I also forgot to mention that adding the files manually seemed to work OK when running it off my system. But when uploading the file and then re-downloading, MacOS would completely stop this from running stating that the app had been tampered with (which it had )  so i was very keen i could build this directly without having to manually add. Probably something that can be fixed when notarising etc, but easier not to have that issue!

Manually is my favorite method. You can build your Mac app before code signing and notarizing it. You should be able to make any adjustments you want before the codesigning process.

As always, we want you to be successful. Let us know how it all goes.

-Mark

[stamatis]

Thanks as always Mark,
I've encountered a further issue which i'm struggling to understand on MacOS and hope you and the team may have some insight...

I now build the app just fine; if i run the app, my software firewall throws a warning this is not codesigned software. But it all works just fine.
Now if i upload to a server and re-download, i get an unbeatable error from MacOS/Gatekeeper saying the 'app is damaged' and offers to 'move this to the bin'.
I presume this is because it's not codesigned/notarised.

So i ventured into the murky world of codeSigning, notarising and stapling the app. Life is too short, so I used the stack mrSignNotarizeHelperV3.livecode that is available from the liveCode lesson on the subject -- promises to make it so much easier.

CodeSigning, Notarisation and Stapling are done just fine - Apple sends me a message it's all successful etc.

However, I now get an error that the libraries couldn't be decrypted. I rebuilt the app and made a copy of it, so i had an untouched app as well as an app to codeSign, notarize and stable. Again that process is fine, but i get the same error, where the untouched app that hasn't been codeSigned works just fine.

Looking at app-bundle/Contents/Resources/_MacOS/CanelaDB/config/config, the config file is actually changed from the untouched to the stapled app. 
I don't understand how or why?

Any insights?
Stam

[mark_talluto]

Hi Stam,

You did everything correctly. The process of code-signing and notarizing is complicated. I can see that you navigated it well.

You can solve the problem you are experiencing by doing the following:

1. 1. On the first run, copy the relevant parts of your project (everything after the starter stack) over to a safe and writeable place.

1. 2. Run the code from the writeable locationa. 

1.   a. Macs: specialFolderPath("asup")

1.   b. Windows: specialFolderPath(26)

1. 3. Have your pathing point to that location when the environment <> "development"

Yep, developing for Apple has become more complex. All of our shipping apps use this method. LCM (Win version) is designed to be a portable app and thus does not use specialFolderPath(26). It instead writes to files and folders next to the .exe.

Should you decide to codesign your Windows apps, then SFP(26) will be useful. Our 2020 Vision product uses that location.

I hope this helps

-Mark

[stamatis]

Hi Stam,

You did everything correctly. The process of code-signing and notarizing is complicated. I can see that you navigated it well.

You can solve the problem you are experiencing by doing the following:

1. 1. On the first run, copy the relevant parts of your project (everything after the starter stack) over to a safe and writeable place.

1. 2. Run the code from the writeable locationa. 

1.   a. Macs: specialFolderPath("asup")

1.   b. Windows: specialFolderPath(26)

1. 3. Have your pathing point to that location when the environment <> "development"

Yep, developing for Apple has become more complex. All of our shipping apps use this method. LCM (Win version) is designed to be a portable app and thus does not use specialFolderPath(26). It instead writes to files and folders next to the .exe.

Should you decide to codesign your Windows apps, then SFP(26) will be useful. Our 2020 Vision product uses that location.

I hope this helps

-Mark

Thanks Mark, 
Not sure i understand why Application Support would work when Documents fails, as both are writeable locations?
Anyway, will try...

[stamatis]

Hi Mark - in way of update, i did get this to work just fine. The destination (Documents vs Application Support) wasn't the issue.

The problem was that i had code accessing LiveCloud from both with the stapled splash stack (doing some initialisation) and within the the 'mainstack' which resides in a writeable location (Documents in this case). I thought that would be fine as all the initialiseCanelaDB and actual code for liveCloud resided in the 'mainstack' and i was just calling functions from there after 'start using' the mainstack from the splash stack. Seems that doesn't work for stapled apps though.

Once i moved all LiveCloud related code out of  the stapled splash stack/standalone, all errors disappeared...

Many thanks
Stam