Extensive loadup scripts revamp #2111
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…inted to by LDEINIT and instead loads the file pointed to by LDEREMCM. LDEINIT remains the file used by greet. Adjusted the medley script to have a new argument -cm (or --rem.cm) which sets LDEREMCM as appropriate before launching lde. Updated the loadup scripts as required to use this new -cm argument when calling medley. Finally, added a new feature to the medley script -cc (or --repeat) whereby which when medley finishes it checks for a nonzero file given as the argument to -cc. If that file exists, medley is run again (i.e., repeated) with LDEREMCM set to that file. This repeats until this file no longer exists or is zero-length. The file can be found as the vale of LDEREPEATCM so that each invocation of medley can modify (or delete) this file so as to change the subsequent run of medley.
…:LOGOUT can now provide an exit code, allowing for better failure modes in loadups. Integrate in the changes to apps loadup from lmm-loadup-fixes branch.
… conditions - uses new exit code capabilities of IL:LOGOUT. 2) Move loadup scripts from MEDLEYDIR/scripts to MEDLEYDIR/scripts/loadup-scripts - leaving behind symlinks to the external scripts -all, -full and -db. 3) Added LOADUP_NOBREAK env variable, which if non-empty string will cause all loadup-scripts to exit with a failure silently and without a break - otherwise failures will cause a break. 4) added -m or --maikodir args to all loadup scripts allowing you to specify which maiko to use - also can be specified thru MAIKODIR env variable.
…loadups/loadup-setup.sh
…ipts/loadups-scripts
…reworking of arg processing to provide for --target command; bug fixes too numerous to mention here
…t redirect to omnibus loadup script
… man page and fix up formatting of man page.
…from aux running automatically. Updated man page accordingly.
|
On Linux Mint 22.1 Cinnamon I ran my usual legacy loadup process under this change, i.e The only thing of notice is on my 5 GHz 13th gen Intel Core i7 PC there's a short delay each time Medley is launched between the various steps of the process. It's completely harmless and no more than a fraction of a second, but enough to be perceived. |
masinter
approved these changes
Apr 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR supersedes PR#1958 as well as PR#1948, which have been closed.
This PR is an extensive revamp of the loadup scripts including the following changes:
New omnibus script scripts/loadup which is meant to be the single interface to the loadup system. The man page for this script can be found here: https://online.interlisp.org/downloads/man_loadup.html
The new loadup script allows you to restart the loadup process from any particular stage - init, mid, lisp, full, etc. For example, you can start the loadup from an existing init-mid.sysout and have it run thru creating the full.sysout. The call for this would be:
./scripts/loadup --target full --start mid. (See man page for all options to the loadup script as well as examples of their use.)In order to facilitate this target/start feature, the loadup workdir has been moved to a single location per MEDLEYDIR, specifically to MEDLEDIR/loadups/build. (Previously, it was different for every invocation in /tmp/loadups-$$)
When restarting the loadup from, say, lisp.sysout, the script will look for the lisp.sysout to start from first in MEDLEYDIR/loadups and then in MEDLEYDIR/loadups/build. If the starting sysout is found in loadups, it will be copied (non-versioned) into the workdir before doing the loadup, overwriting whatever is already there.
There is now a lock (MEDLEYDIR/loadups/build/lock) that prevents concurrent loadups from running (and very occassionally needs to be manually removed).
At successful completion of a loadup run, the created .sysouts and other files are moved into the loadups directory as before (unless the --nocopy option is specified). BUT the build files - including the dribble files - are left in the working directory (i.e., in loadups/build)
The loadup script now supersedes the former loadup-all.sh, loadup-full.sh and loadup-db.sh scripts. But I have left in their place scripts that call the new loadup script with the right options. loadup-all.sh is now just a link to the single loadup script, who's options are (hopefully) a superset of loadup-all.sh.
All of the worker scripts (e.g., loadup-full-from-lisp.sh) have been updated to use better mechanisms to catch and report errors, including the new (LOGOUT T EXITCODE) feature. The main script has been updated to better catch errors in these worker scripts when they do happen.
You can now specify a MAIKODIR for the loadup, either using the MAIKODIR env variable or thru the
--maikodircommand line option.All of the loadup scripts have been moved down one level into scripts/loadups. But there are links from the scripts/ directory into the scripts/loadups directory for all of the top-level legacy scripts - loadup-all.sh, loadup-db.sh, loadup-full.sh so that you do not have to change your own scripts unless you need to use some of the new features. More importantly, none of the github workflows need to change right now.
The SYSOUTCOMMITS mechanism had to be changed since there is no guarantee that e.g., lisp.sysout and full.sysout are built on the same commit. So (ASSOC 'MEDLEY SYSOUTCOMMITS) now returns an assoc list of sysouts, e.g., ((INIT "aaa")(LISP "bbb")(FULL "ccc")(APPS "ddd")), showing the commits for the various layers of the loadup.
The new loadup scheme allows different sysouts to be created from different commits, so there is an issue with RDSYS (and RDSYS.LCOM) being out of sync with one or more of the sysout in loadups. RDSYS(.LCOM) are copied into library when ever a loadup of the Init stage completes successfully (unless the -nocopy option is specified). The only way to solve this issue when it arises is to do a complete loadup from starter.sysout to full.sysout (or apps.sysout) to ensure evrything is built on the same commit.