Skip to content

fix(aws-healthomics): correct namespaced run parameters in WDL migration example, and CREATED workflow status - #177

Merged
nadetastic merged 1 commit into
kirodotdev:mainfrom
hmkim:fix/wdl-migration-run-parameters
Aug 6, 2026
Merged

fix(aws-healthomics): correct namespaced run parameters in WDL migration example, and CREATED workflow status#177
nadetastic merged 1 commit into
kirodotdev:mainfrom
hmkim:fix/wdl-migration-run-parameters

Conversation

@hmkim

@hmkim hmkim commented Aug 4, 2026

Copy link
Copy Markdown

What

Three factual corrections to the aws-healthomics steering docs:

  1. The WDL migration guide's S3 Input (Before/After) example keeps the workflow namespace on run parameter keys, contradicting workflow-development.md in the same Power.
  2. troubleshooting.md tells the agent to wait for a workflow status of CREATED, which is not a HealthOmics workflow status.
  3. migration-guide-for-wdl.md mixes GB and GiB within the same file.

Documentation only. No changes to the Nextflow guide, the ECR pull-through cache SOP, or any resource limit values.

1. Namespaced run parameters in the migration example

Why it's wrong — the same Power says the opposite. workflow-development.md:71-85:

Workflow parameters MUST NOT be namespaced:

Wrong: { "MyWorkflow.input_file": "s3://bucket/path/to/input.vcf" }

But migration-guide-for-wdl.md:232-239 shows this as the migrated "After" state:

// After
{ "WorkflowName.reference_fasta": "s3://bucket/references/.../reference.fasta" }

The Before/After changes the path to S3 but leaves the namespace in place. The file this SOP produces is a run parameters file — migration-guide-for-wdl.md:135 and :177 create healthomics.inputs.json, and running-a-workflow.md pre-condition 2 verifies "a parameters.json or inputs.json exists with valid, accessible inputs."

Why it matters more than a wrong example usually wouldStartRun does not reject namespaced keys, so this does not fail loudly.

Measured (2026-07, ap-northeast-2, account IDs masked below):

run 9239009, workflow wdl-migration-demo-coerce-strict

    parameters sent:
        "coerce_demo.thread_count_str": "4"
        "coerce_demo.mem_gb_float":     "3"

    StartRun:   accepted
    run status: COMPLETED

The workflow in that probe declares defaults for both parameters, which appears to be what makes the namespaced keys tolerable: the bare parameters are optional, so nothing is reported missing, and the unknown namespaced keys do not trip a check. The defaults run, the run reports COMPLETED, and the numbers in the output are not the numbers that were submitted.

So whether this surfaces at all depends on whether the workflow declares defaults. All-required inputs fail loudly (the bare name is missing). Inputs with defaults were accepted silently here. I've worded the steering change as conditional (MAY be accepted silently) rather than describing StartRun's validation rule, since I only measured the one case.

The audience for this file makes it worse: readers migrating from Cromwell arrive holding an inputs file where every key is namespaced, since that is Cromwell's required format. This is the one group most likely to copy the "After" block verbatim.

Change — the "After" example uses a bare key. The rule and the silent-acceptance caveat go in Phase 4 step 4 and its Done WHEN, since that is where the file is created, with a cross-reference to workflow-development.md. I kept ## Technical Patterns to a code example plus one sentence, matching the two sibling subsections.

2. CREATED is not a workflow status

troubleshooting.md:9:

IF a workflow fails to reach CREATED status, check these causes in order:

Measured: CreateWorkflow returns HTTP 200 with status CREATING, and the workflow then transitions to ACTIVE or FAILED. There is no CREATED.

The asynchronous transition is worth stating explicitly too. Re-measured today in us-west-2 with an intentionally invalid definition:

$ aws omics create-workflow --name aho-pr177-syntaxfail-probe --engine WDL ...
{ "id": "4737558", "status": "CREATING" }        <- HTTP 200, not a rejection

$ aws omics get-workflow --id 4737558            (after polling)
{ "status": "FAILED",
  "statusMessage": "(main.wdl Ln 12 Col 3) Unknown type THIS_IS_NOT_WDL ..." }

An invalid definition is accepted at submission time and reported as FAILED only on a later GetWorkflow. A script that treats the 200 as success reports a broken workflow as registered.

Two other probes in the same session are worth reporting, because they qualify the SOP's own premise. A definition with no version declaration registered ACTIVE, not FAILED:

$ aws omics get-workflow --id 3334792
{ "status": "ACTIVE",
  "statusMessage": "main.wdl (Ln 1, Col 1) MissingVersion, document should
                    declare WDL version; draft-2 assumed ..." }

So migration-guide-for-wdl.md:14's "draft-2 is NOT supported" does not mean "rejected at creation" — the service assumes draft-2 and proceeds, recording it as a warning on an ACTIVE workflow. I have not changed that line, since I don't know whether the intent is "unsupported" in the sense of unvalidated-and-unrecommended. But it is why the added note says invalid definitions surface as FAILED rather than singling out draft-2, and why it also tells the agent to read statusMessage on ACTIVE workflows.

ChangeCREATEDACTIVE, a sentence on the asynchronous transition, and a note to read statusMessage even when the status is ACTIVE. workflow-development.md Step 2 says "Call GetAHOWorkflow to verify the workflow was created successfully" without saying which status to wait for, so the polling note is added there.

3. GB / GiB mixed in one file

Line Text
migration-guide-for-wdl.md:57 Min 2 vCPUs / 4 GiB memory. Max 96 vCPUs / 768 GiB memory.
migration-guide-for-wdl.md:68 memory: "8 GiB"
migration-guide-for-wdl.md:76 ≥2 vCPU, ≥4 GB
migration-guide-for-wdl.md:207 memory: "8 GB"

The two GiB occurrences are the ones stating the limits, so :76 and :207 are aligned to GiB. GB and GiB differ by 7.4%, which is enough to move a borderline task into OOM, and an example that mixes units teaches the reader to mix them.

Deliberately not changed: the numeric limits themselves, and the GB/GiB inconsistency between this file and migration-guide-for-nextflow.md:44 (Min 2 vCPUs / 4 GB). workflow-development.md:40 states a third figure (at least 1 GB memory and 1 CPU), and I don't know which is authoritative — happy to file that separately if useful.

Measurement conditions

The run parameter probe: 2026-07, ap-northeast-2, two accounts (one with no pre-existing HealthOmics resources), miniwdl 1.15.0 / Python 3.12.13. The creation probes above (3334792, 4737558): re-measured 2026-08-04 in us-west-2 to confirm the behavior had not changed. AWS account IDs are masked as 123456789012; run and workflow IDs are left as-is since they are meaningless without the account.

…status

The WDL migration guide's S3 Input Before/After example kept the Cromwell
workflow namespace on run parameter keys, contradicting
workflow-development.md ("Workflow parameters MUST NOT be namespaced") in
the same Power. StartRun does not reliably reject namespaced keys: where the
workflow declares a default, the key can be accepted silently and the
default runs instead of the submitted value, so the run reports COMPLETED
with results that do not reflect its inputs.

troubleshooting.md directed the agent to wait for a `CREATED` workflow
status, which does not exist. CreateWorkflow returns `CREATING` and the
workflow then transitions to `ACTIVE` or `FAILED`, so note the asynchronous
transition here and in workflow-development.md Step 2. Also note that
`statusMessage` carries warnings on workflows that DO reach `ACTIVE`, such
as a missing version declaration causing draft-2 to be assumed.

Also align the two `GB` occurrences in migration-guide-for-wdl.md to the
`GiB` used by the limits statement in the same file. Resource limit values
are unchanged.
@hmkim
hmkim force-pushed the fix/wdl-migration-run-parameters branch from e4afd15 to df22cf6 Compare August 4, 2026 14:59
@nadetastic
nadetastic merged commit c9cf2ee into kirodotdev:main Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants