Skip to content

Jobs

Why a job ended the way it did (Job.status, its end reason). How to read an explanation, and how to ask for one (openflowsheet explain <code>, the explain operation), is on Messages.

cancel_requested: The job was cancelled on request

Kind: Notice: not a failure.

  • Summary: The job ended: it was cancelled on request.
  • Why: Someone with the right to do so cancelled the job (cancel_job) while it was queued or running.
  • Do: Submit the job again if you still need its result.

Example:

The job ended: it was cancelled on request.
  Why:     Someone with the right to do so cancelled the job (cancel_job) while it was queued or running.
  Do:      Submit the job again if you still need its result.
  Kind:    Notice: not a failure.
  Code:    cancel_requested (openflowsheet explain cancel_requested)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#cancel_requested

cancelled: The job was cancelled

Kind: Notice: not a failure.

  • Summary: The job <job> was cancelled before it finished.
  • Why: A job is cancelled when someone asked for it (cancel_job), when the server running it shut down, or when the program running it was interrupted from the keyboard. The ending reason says which.
  • Do: Submit the job again if you still need its result.
  • See also: cancel_requested, server_shutdown, keyboard_interrupt

Example:

The job was cancelled before it finished.
  Why:     A job is cancelled when someone asked for it (cancel_job), when the server running it shut down, or when the program running it was interrupted from the keyboard. The ending reason says which.
  Do:      Submit the job again if you still need its result.
  Kind:    Notice: not a failure.
  Code:    cancelled (openflowsheet explain cancelled)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#cancelled

completed: The job ran its operation to the end

Kind: Notice: not a failure.

  • Summary: The job <job> completed: its operation ran to the end.
  • Why: A completed job ran its operation and stored its results. For a solve this does not mean that the solve converged: the job's result gives the solve's outcome and its verification status.
  • Do: Read the job's result; for a solve, check its outcome and verification status before relying on it.
  • See also: CONVERGED, VERIFIED

Example:

The job completed: its operation ran to the end.
  Why:     A completed job ran its operation and stored its results. For a solve this does not mean that the solve converged: the job's result gives the solve's outcome and its verification status.
  Do:      Read the job's result; for a solve, check its outcome and verification status before relying on it.
  Kind:    Notice: not a failure.
  Code:    completed (openflowsheet explain completed)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#completed

failed: The job ended without completing its operation

Kind: Notice: not a failure.

  • Summary: The job <job> failed: it ended without completing its operation.
  • Why: A job fails when its operation raised an error, when the verifier refused to judge its result, when the process that ran it or the program that owned it was lost, or when the project's database reported an error. The job's ending reason says which, and its error says what happened.
  • Do: Read the job's error and ending reason; each has its own explanation.
  • See also: operation_error, verifier_refused, worker_lost, owner_lost, store_error

Example:

The job failed: it ended without completing its operation.
  Why:     A job fails when its operation raised an error, when the verifier refused to judge its result, when the process that ran it or the program that owned it was lost, or when the project's database reported an error. The job's ending reason says which, and its error says what happened.
  Do:      Read the job's error and ending reason; each has its own explanation.
  Kind:    Notice: not a failure.
  Code:    failed (openflowsheet explain failed)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#failed

keyboard_interrupt: The job was cancelled by a keyboard interrupt

Kind: Notice: not a failure.

  • Summary: The job ended: the program running it was interrupted from the keyboard.
  • Why: The program running the job received an interrupt (Ctrl-C), so it cancelled the job before stopping.
  • Do: Submit the job again if you still need its result.

Example:

The job ended: the program running it was interrupted from the keyboard.
  Why:     The program running the job received an interrupt (Ctrl-C), so it cancelled the job before stopping.
  Do:      Submit the job again if you still need its result.
  Kind:    Notice: not a failure.
  Code:    keyboard_interrupt (openflowsheet explain keyboard_interrupt)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#keyboard_interrupt

operation_completed: Operation completed

Kind: Notice: not a failure.

No plain-language explanation is written yet; the summary is the technical one.

  • Summary: The job ended: operation completed.

operation_error: Operation error

Kind: Notice: not a failure.

No plain-language explanation is written yet; the summary is the technical one.

  • Summary: The job ended: operation error.

owner_lost: The program that accepted the job stopped before it finished

Kind: Internal error in OpenFlowsheet: please report it.

  • Summary: The job ended: the program that accepted it stopped, or lost the job, before the job finished.
  • Why: A job belongs to the program (a Python session, the CLI or a server) that accepted it. That program ended or lost its hold on the job while the job was queued or running, so the next program to open the project marked the job failed.
  • Do: Submit the job again from a program that stays open until the job ends, for example by waiting for it (wait_job) before the program exits.

Example:

The job ended: the program that accepted it stopped, or lost the job, before the job finished.
  Why:     A job belongs to the program (a Python session, the CLI or a server) that accepted it. That program ended or lost its hold on the job while the job was queued or running, so the next program to open the project marked the job failed.
  Do:      Submit the job again from a program that stays open until the job ends, for example by waiting for it (wait_job) before the program exits.
  Kind:    Internal error in OpenFlowsheet: please report it.
  Code:    owner_lost (openflowsheet explain owner_lost)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#owner_lost

queued: Queued

Kind: Notice: not a failure.

No plain-language explanation is written yet; the summary is the technical one.

  • Summary: The job ended: queued.

running: Running

Kind: Notice: not a failure.

No plain-language explanation is written yet; the summary is the technical one.

  • Summary: The job ended: running.

server_shutdown: The job was cancelled because its server shut down

Kind: Notice: not a failure.

  • Summary: The job ended: the server running it shut down before it finished.
  • Why: When the server shuts down cleanly it cancels its queued and running jobs, so none is left half-finished. Nothing was wrong with the job.
  • Do: Submit the job again once the server is back; it may succeed unchanged.

Example:

The job ended: the server running it shut down before it finished.
  Why:     When the server shuts down cleanly it cancels its queued and running jobs, so none is left half-finished. Nothing was wrong with the job.
  Do:      Submit the job again once the server is back; it may succeed unchanged.
  Kind:    Notice: not a failure.
  Code:    server_shutdown (openflowsheet explain server_shutdown)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#server_shutdown

store_error: The project database failed while the job ran

Kind: Internal error in OpenFlowsheet: please report it.

  • Summary: The job ended: the project's database reported an error while the job ran.
  • Why: A job reads and writes the project's database (project.sqlite3) as it runs. The database refused an operation, for example because the disk is full, the file is not writable, or another program holds it locked.
  • Do: Check that the project's directory is writable and its disk has free space, then submit the job again. If the error recurs with both in order, report it with the server log.

Example:

The job ended: the project's database reported an error while the job ran.
  Why:     A job reads and writes the project's database (project.sqlite3) as it runs. The database refused an operation, for example because the disk is full, the file is not writable, or another program holds it locked.
  Do:      Check that the project's directory is writable and its disk has free space, then submit the job again. If the error recurs with both in order, report it with the server log.
  Kind:    Internal error in OpenFlowsheet: please report it.
  Code:    store_error (openflowsheet explain store_error)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#store_error

timed_out: The job ran out of wall time

Kind: Notice: not a failure.

  • Summary: The job <job> timed out: it used its wall-time budget before it finished.
  • Why: Every job may carry a wall-time budget: the request's, or else the credential's default or ceiling. This job reached it and was stopped before it finished.
  • Do: Submit the job again with a larger wall-time budget, within the credential's ceiling.
  • See also: wall_time_exhausted, budget_exceeds_ceiling

Example:

The job timed out: it used its wall-time budget before it finished.
  Why:     Every job may carry a wall-time budget: the request's, or else the credential's default or ceiling. This job reached it and was stopped before it finished.
  Do:      Submit the job again with a larger wall-time budget, within the credential's ceiling.
  Kind:    Notice: not a failure.
  Code:    timed_out (openflowsheet explain timed_out)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#timed_out

verifier_refused: Verifier refused

Kind: Notice: not a failure.

No plain-language explanation is written yet; the summary is the technical one.

  • Summary: The job ended: verifier refused.

wall_time_exhausted: The job used its wall-time budget

Kind: Notice: not a failure.

  • Summary: The job ended: it used its wall-time budget before it finished.
  • Why: The job carried a wall-time budget, the request's or the credential's default, and was stopped when it reached it.
  • Do: Submit the job again with a larger wall-time budget, within the credential's ceiling.

Example:

The job ended: it used its wall-time budget before it finished.
  Why:     The job carried a wall-time budget, the request's or the credential's default, and was stopped when it reached it.
  Do:      Submit the job again with a larger wall-time budget, within the credential's ceiling.
  Kind:    Notice: not a failure.
  Code:    wall_time_exhausted (openflowsheet explain wall_time_exhausted)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#wall_time_exhausted

worker_lost: The job's worker process stopped without a result

Kind: Internal error in OpenFlowsheet: please report it.

  • Summary: The job ended: the process that ran it could not be started or stopped without a result.
  • Why: Each job runs in a worker process. That process failed to start, or ended before it reported a result: it crashed, was killed, or ran out of memory. Nothing in the flowsheet is known to be wrong.
  • Do: Submit the job again. If it stops the same way again, report it with the job's log artifact, which the job's error names when one was written.

Example:

The job ended: the process that ran it could not be started or stopped without a result.
  Why:     Each job runs in a worker process. That process failed to start, or ended before it reported a result: it crashed, was killed, or ran out of memory. Nothing in the flowsheet is known to be wrong.
  Do:      Submit the job again. If it stops the same way again, report it with the job's log artifact, which the job's error names when one was written.
  Kind:    Internal error in OpenFlowsheet: please report it.
  Code:    worker_lost (openflowsheet explain worker_lost)
  Docs:    https://computational-chemical-engineering.github.io/openflowsheet/messages/jobs/#worker_lost