Grace LogoGrace

Virtual Paths & Job I/O

In Grace, workflows often involve multiple steps that need to pass data to each other.

This data might:

  • Live on the mainframe as a dataset.
  • Reside on your local machine as a file.
  • Be accessible as a resource in the cloud.
  • Be a temporary artifact created by one job and consumed by another.

To manage this complexity, Grace uses virtual paths within the inputs and outputs sections of your job definitions.

A virtual path is a URI-like string (e.g., src://mycode.cbl, zos-temp://report.txt) that tells Grace about the nature and intended location or source of a piece of data. Grace then resolves these virtual paths to actual physical locations (like a z/OS dataset name or a local file path) at the appropriate time.

This system allows you to define how data flows through your workflow in a declarative way, without having to manage file handoff and lifecycles manually. Grace uses these definitions to:

  • Wire data dependencies between jobs.
  • Generate appropriate JCL DD statements for mainframe jobs.
  • Manage the lifecycle of temporary data (creation and cleanup).
  • Facilitate data movement for jobs resources across platforms.

Use inputs to tell a job what data it needs. Use outputs to declare what data a job produces and expose it for downstream jobs to consume. Use virtual paths in the path field to describe that data to Grace.


Virtual path prefixes

Grace uses several distinct prefixes (schemes) in the path field to indicate how a data resource should be treated. Each prefix has specific implications for where Grace looks for data, how it's managed, and how it's made available to your jobs.

The following sections detail each virtual path prefix:


inputs and outputs

As introduced in Jobs, you can specify inputs and outputs arrays for jobs to consume or produce. These arrays define the "data contract" for each job.

Each item within the inputs or outputs array is an object that describes a single piece of data. The two most important fields for each item are:

name

(String, required)

A logical name for this data item.

  • For z/OS job types (compile, linkedit, execute), this name directly translates to the DDName in the generated JCL (e.g. SYSIN, SYSLIN, CUSTFILE)
  • For shell jobs, this name is used to form the environment variables (e.g. GRACE_INPUT_SYSIN, GRACE_OUTPUT_SYSLIN) that provide local paths to the data for your script.

path

(String, required)

The virtual path that specifies the source (for inputs) or intended destination/identifier (for outputs) of the data. The prefix of this path (e.g. src://, zos://, zos-temp://) determines how Grace interprets and handles it.


Example inputs and outputs structure:

jobs:
  - name: MY_SHELL_JOB
    type: shell
    inputs:
      - name: EBCDIC_TEXT_FILE
        path: zos://USER.DATA.EBCDIC
        encoding: text # Download as text, convert EBCDIC to local encoding
 
      - name: BINARY_MODULE
        path: zos://USER.LOADLIB(MYPROG)
        encoding: binary # Download as raw binary
 
      - name: DEFAULT_BEHAVIOR_FILE
        path: zos://USER.SOME.OTHER.BIN # No encoding defaults to binary
 
    outputs:
      - name: SOME_OUTPUT
        path: local-temp://output.dat
        # ... other output specific fields like keep, dcb, space ...
    # ... other job fields

Additional parameters

While name and path are always present, additional parameters can be specified within an input or output item to provide finer control, particularly for how JCL DD statements are generated for z/OS datasets and how temporary resources are managed.

These parameters are typically most relevant when the path refers to a zos:// or zos-temp:// resource, or when defining the characteristics of an output dataset to be created on z/OS.

For details on how these and other resolved values are exposed when writing custom JCL templates, see the JCL Templating Guide.


disp

inputs:
  - name: OLDMAST
    path: zos://PROD.MASTER
    disp: OLD # Ensure exclusive access
 
outputs:
  - name: NEWMAST
    path: zos://PROD.NEWMASTER
    disp: "(NEW,CATLG,KEEP)" # Create, catalog, and keep on normal/abnormal end
  • Type: String

  • Optional: Yes

  • Applies to: inputs and outputs (for z/OS datasets)

  • Description: Specifies the JCL DISP= (disposition) subparameter for the DD statement. This controls the status of the dataset before and after the job step.

  • Grace's default behavior:

    • For inputs: Defaults to SHR (e.g., DISP=SHR).
    • For outputs using zos-temp:// or new zos:// datasets: Defaults to (NEW,CATLG,DELETE) (or (NEW,CATLG,CATLG) if keep: true is set for a zos-temp:// output).
    • For outputs using zos:// pointing to an existing dataset you intend to modify: You might specify MOD (e.g., DISP=MOD or DISP=(MOD,KEEP)).

dcb

outputs:
  - name: REPORTFILE
    path: zos-temp://daily.report
    dcb: "RECFM=FBA,LRECL=133,BLKSIZE=13300"
  • Type: String

  • Optional: Yes

  • Applies to: outputs (primarily for z/OS datasets being created or written to)

  • Description: Specifies the JCL DCB= (Data Control Block) subparameters for the DD statement, defining the dataset's characteristics.

  • Format: A string containing comma-separated DCB attributes (e.g., "RECFM=FB,LRECL=80,BLKSIZE=27920").

  • Grace's default behavior: For zos-temp:// outputs, Grace uses a sensible default (e.g., (RECFM=FB,LRECL=80)) if not specified. For zos:// outputs where a new dataset is being created, providing DCB attributes is often necessary.


space

outputs:
  - name: BIGDATAOUT
    path: zos-temp://large.extract
    space: "(CYL,(100,50),RLSE)" # Allocate a larger temporary dataset
  • Type: String

  • Optional: Yes

  • Applies to: outputs (primarily for z/OS datasets being created)

  • Description: Specifies the JCL SPACE= parameter for allocating space to a new dataset on z/OS.

  • Format: A string representing the JCL SPACE parameter (e.g., "(TRK,(10,5),RLSE)", "(CYL,(1,1))").

  • Grace's default behavior: For zos-temp:// outputs, Grace uses a modest default space allocation (e.g., (TRK,(5,5),RLSE)) if not specified. For zos:// outputs where a new dataset is being created, providing appropriate space parameters is crucial.


keep

outputs:
  - name: DEBUG_OBJ
    path: zos-temp://myprog.debug.obj
    keep: true # Do not delete this object deck after the workflow
  • Type: Boolean

  • Optional: Yes

  • Applies to: outputs using zos-temp:// or local-temp:// paths.

  • Default: false

  • Description: Controls whether Grace automatically cleans up (deletes) this temporary resource after the workflow execution completes.

    • If true: Grace will not delete the temporary z/OS dataset or local file during its standard cleanup procedures. This is useful for debugging or if an intermediate "temporary" resource needs to be inspected or used outside of Grace after the workflow.
    • If false (default): The temporary resource is subject to deletion based on the global config.cleanup settings (which default to deleting on workflow success).
  • Effect on DISP for zos-temp://:

    When keep: true is set for a zos-temp:// output, Grace will override the DISP parameter in the generated JCL to (NEW,CATLG,CATLG) to ensure the dataset is kept even if the step ABENDs (after successful creation and cataloging in the step). It is recommended to use either disp or keep fields, but not both.


encoding

inputs:
  - name: SOURCE_CODE
    path: zos://MY.COBOL.LIB(PROG01)
    encoding: text
 
  - name: LOAD_MODULE
    path: zos://MY.LOAD.LIB(PROG01)
    # omit encoding for binary by default
 
  - name: VSAM_BACKUP
    path: zos://MY.VSAM.BACKUP.FILE # Defaults to binary, suitable for VSAM
    encoding: binary # Explicitly setting it also works
  • Type: String
  • Optional: Yes
  • Default: "binary" (if omitted or empty)
  • Applies to: inputs where the path is zos:// or zos-temp:// and the data is being resolved by a local process
  • Description: Specifies how Grace should handle character encoding when routing the resource off of the mainframe.
  • Available values:
    • binary
      • Performs a raw, byte-for-byte transfer. No character set conversion is attempted.
      • Use case: Essential for true binary files like load modules, VSAM files with packed/binary fields, compressed data, or when you need the file to retain the exact EBCDIC representation.
    • text
      • Treats the mainframe resource as a text file (typically EBCDIC) and converts it to a standard readable text format on the local machine (usually UTF-8 or the system's default ASCII-compatible encoding).
      • Use case: Ideal for COBOL source code, JCL members, report files, parameter files, or any other text-based dataset that needs to be human-readable or processed by standard text tools.

On this page