Consider MODULE Directive to Implement Sub-Module Pattern

[TekWizely]

Consider this Runfile

# justprep/test/Runfile

module_aaa := "modules/aaa.run"
module_bbb := "modules/bbb.run"

EXPORT module_aaa, module_bbb

##
# Say hi
hello:
  echo "Hello World"

##
# List all tasks including the modules
list_all:
  run list
  run aaa
  run bbb

##
# Module aaa
aaa:
  run -r ${module_aaa} ${@}

##
# Module bbb
bbb:
  run -r ${module_bbb} ${@}

and two others aaa.run and bbb.run both in a modules directory.

# .../modules/aaa.run

##
# Module aaa magic
magic:
  echo 'Do you believe in Magic?'

##
# Do something magical in aaa
xyzzy:
  echo "Magic from aaa ..."
  echo "... watch me pull a ${1} out of my hat."

... and ...

# .../modules/bbb.run

##
# Module bbb magic
magic:
  echo 'Do you believe in Magic?'

##
# Do something magical in bbb
xyzzy:
  echo "Magic from bbb ..."
  echo "... watch me pull a ${1} out of my hat."

Here is a console session using those three files:

13:51:20 3.1.2p20 support_run s5:test $ run
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/Runfile

Commands:
  list        (builtin) List available commands
  help        (builtin) Show help for a command
  version     (builtin) Show run version
  hello       Say hi
  list_all    List all tasks including the modules
  aaa         Module aaa
  bbb         Module bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:30 3.1.2p20 support_run s5:test $ run list_all
Commands:
  list        (builtin) List available commands
  help        (builtin) Show help for a command
  version     (builtin) Show run version
  hello       Say hi
  list_all    List all tasks including the modules
  aaa         Module aaa
  bbb         Module bbb
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/aaa.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module aaa magic
  xyzzy      Do something magical in aaa

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/bbb.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module bbb magic
  xyzzy      Do something magical in bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:41 3.1.2p20 support_run s5:test $ run aaa
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/aaa.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module aaa magic
  xyzzy      Do something magical in aaa

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:45 3.1.2p20 support_run s5:test $ run bbb
using runfile: /Users/dewayne/Documents/sandbox/git_repos/madbomber/justprep/working/support_run/test/modules/bbb.run

Commands:
  list       (builtin) List available commands
  help       (builtin) Show help for a command
  version    (builtin) Show run version
  magic      Module bbb magic
  xyzzy      Do something magical in bbb

Usage:
       run <command> [option ...]
          (run <command>)
  or   run help <command>
          (show help for <command>)

see 'run --help' for more information
13:51:52 3.1.2p20 support_run s5:test $ run aaa magic
Do you believe in Magic?
13:51:58 3.1.2p20 support_run s5:test $ run bbb magic
Do you believe in Magic?
13:52:04 3.1.2p20 support_run s5:test $ run aaa xyzzy rabbit
Magic from aaa ...
... watch me pull a rabbit out of my hat.
13:52:14 3.1.2p20 support_run s5:test $ run bbb xyzzy dinosaur
Magic from bbb ...
... watch me pull a dinosaur out of my hat.
13:52:33 3.1.2p20 support_run s5:test $

What I do in justprep is to turn a line that looks like this ...

module aaa := "modules/aaa.run

into lines that look like this ...

module_aaa := "modules/aaa.run
EXPORT module_aaa

##
# Module aaa
aaa:
  run -r ${module_aaa} ${@}

Originally posted by @MadBomber in #47 (comment)

[TekWizely]

@MadBomber wrote:

aaa:
  run -r ${module_aaa} ${@}

consider that Fake Module Convention as a possible feature for the next release.

So this is very clever ! In fact i think it deserves to be called "Sub-Module Pattern" - no "Fake" about it :)

Indeed we could consider introducing a MODULE directive.

( fun fact: commands actually accept an optional COMMAND (or CMD) directive when defining them )

Honestly, though I don't know if the effort would yield enough value - Let's discuss ...

Your pattern is pretty easy to implement. you can imagine moving the module name export to the command's doc block to make it a bit more compact:

##
# module aaa. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "aaa help" to learn more
# export RUNMODULE := "modules/aaa.run"
aaa:
    run -r ${RUNMODULE} ${@}

##
# module bbb. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "bbb help" to learn more
# export RUNMODULE := "modules/bbb.run"
bbb:
    run -r ${RUNMODULE} ${@}

A couple of quirks in the pattern:

• The RUNMODULE path is resolved relative the user's PWD vs the primary Runfile's location. This can be a bug when invoking a runfile from outside your current PWD. One way to address it:

export RUNFILE := ${.RUNFILE}

##
# module aaa. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "aaa help" to learn more
# export RUNMODULE := "modules/aaa.run"
aaa:
    run -r ${RUNFILE%/*}/${RUNMODULE} ${@}

However it increases overall verbosity

• When getting a list of commands from the RUNMODULE, the list includes run's builtin commands:

run aaa list
Commands:
 list       (builtin) List available commands
 help       (builtin) Show help for a command
 version    (builtin) Show run version
 magic      Module aaa magic
 xyzzy      Do something magical in aaa

OK now let's see what a MODULE directive might be like:

## HIGH-LEVEL DESCRIPTION OF THE MODULE.
MODULE aaa "modules/aaa.run"

MODULE bbb "modules/bbb.run"

Some things I see:

• Run could generate a consistent help text
• High-level description is optional, Run could still generate the "module xxx. use xxx help to learn more" even if no description is given
• This is definitely less verbose, and significantly less verbose when no module description is given
• Run could pre-read the module Runfile :
  ** Confirming it exists and parses
  ** Making its commands available as part of the module-level help text
  ** NOTE: We probably would only read the file if we were interacting with the module
• Invoking help could be "run help aaa magic" (or maybe "aaa.magic" ?) vs "run aaa help magic", which better-matches the standard help pattern for commands.
• Per the INCLUDE processing logic, Run would treat the module Runfile's path as being relative to the primary Runfile's parent directory

So there are some benefits, but adding the support wouldn't be trivial imo and the overall gains might not be significantly better than your pattern.

I'll keep grinding on it though and would appreciate any further thoughts you have.

in the meantime, I encourage you to submit a PR updating Run's README with a section on your Sub-Module Pattern, perhaps under the Invoking Other Commands & Runfiles
section.

It may even be worth a blog post.

Thank you for participating in my project !

-TW

[MadBomber]

quickly,,, path to module should also support absolute path. agree when relative it’s to the invoking rundile. modules are very useful re encapsulation and consistency. for example i have a module that handles semantic versioning and nothing else. Dewayne o-*

…

On Jul 3, 2022, at 11:19 PM, TekWizely ***@***.***> wrote:  @MadBomber wrote: aaa: run -r ${module_aaa} ${@} consider that Fake Module Convention as a possible feature for the next release. So this is very clever ! In fact i think it deserves to be called "Sub-Module Pattern" - no "Fake" about it :) Indeed we could consider introducing a MODULE directive. ( fun fact: commands actually accept an optional COMMAND (or CMD) directive when defining them ) Honestly, though I don't know if the effort would yield enough value - Let's discuss ... Your pattern is pretty easy to implement. you can imagine moving the module name export to the command's doc block to make it a bit more compact: ## # module aaa. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "aaa help" to learn more # export RUNMODULE := "modules/aaa.run" aaa: run -r ${RUNMODULE} ${@} ## # module bbb. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "bbb help" to learn more # export RUNMODULE := "modules/bbb.run" bbb: run -r ${RUNMODULE} ${@} A couple of quirks in the pattern: The RUNMODULE path is resolved relative the user's PWD vs the primary Runfile's location. This can be a bug when invoking a runfile from outside your current PWD. One way to address it: export RUNFILE := ${.RUNFILE} ## # module aaa. HIGH-LEVEL DESCRIPTION OF THE MODULE. use "aaa help" to learn more # export RUNMODULE := "modules/aaa.run" aaa: run -r ${RUNFILE%/*}/${RUNMODULE} ${@} However it increases overall verbosity When getting a list of commands from the RUNMODULE, the list includes run's builtin commands: run aaa list Commands: list (builtin) List available commands help (builtin) Show help for a command version (builtin) Show run version magic Module aaa magic xyzzy Do something magical in aaa OK now let's see what a MODULE directive might be like: ## HIGH-LEVEL DESCRIPTION OF THE MODULE. MODULE aaa "modules/aaa.run" MODULE bbb "modules/bbb.run" Some things I see: Run could generate a consistent help text High-level description is optional, Run could still generate the "module xxx. use xxx help to learn more" even if no description is given This is definitely less verbose, and significantly less verbose when no module description is given Run could pre-read the module Runfile : ** Confirming it exists and parses ** Making its commands available as part of the module-level help text ** NOTE: We probably would only read the file if we were interacting with the module Invoking help could be "run help aaa magic" (or maybe "aaa.magic" ?) vs "run aaa help magic", which better-matches the standard help pattern for commands. Per the INCLUDE processing logic, Run would treat the module Runfile's path as being relative to the primary Runfile's parent directory So there are some benefits, but adding the support wouldn't be trivial imo and the overall gains might not be significantly better than your pattern. I'll keep grinding on it though and would appreciate any further thoughts you have. in the meantime, I encourage you to submit a PR updating Run's README with a section on your Sub-Module Pattern, perhaps under the Invoking Other Commands & Runfiles section. It may even be worth a blog post. Thank you for participating in my project ! -TW — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[nikolay]

I was just about to post an issue regarding this. It would be nice to follow the modern CLI concepts and be able to run commands like:

$ run terraform upgrade
$ run terraform module list
$ run terraform help
$ run terraform module help

and similar.

I already have a separate Runfile per root module, but having to use terraform-upgrade, terraform-module-list, and so on gets the list of possible commands super crowded.

One workaround could be the special treatment of dashes in the command names. So, you still write terraform-upgrade and terraform-module-list in the Runfile, but the CLI finds these common prefixes and allows using spaces instead and uses them to display help better. Pretty much like Git CLI works.