Monitoring and debugging
Use the dltHub CLI and the Web UI at app.dlthub.com to monitor pipeline health, inspect logs, and diagnose failures.
Check workspace status
Retrieve a workspace summary from the CLI:
dlthub workspace info
The command returns the workspace name, job count, latest run status, and the latest deployment and configuration versions. See dlthub workspace info.
Inspect jobs
dlthub job list # all jobs
dlthub job list "tag:ingest" # jobs matching selector
dlthub job list batch # only batch jobs
dlthub job info <name> # details for one job
dlthub job show <name> # open the job page in the dashboard
Inspect runs
List, inspect, and open runs from the CLI:
# list runs (optionally filter by job name or selector)
dlthub job runs list [name_or_selector] [--running]
# exit status, timing, and metadata of a run (defaults to latest)
dlthub job runs info <name> [run#]
# open the run page in the web dashboard
dlthub job runs show <name> [run#]
View logs
From the CLI
Display logs for the latest run of a job:
dlthub job logs my_pipeline.py
Display logs for a specific run number:
dlthub job logs my_pipeline.py 3
To stream logs in real time while a run is in progress, pass --follow, or supply it directly to dlthub run.
For all options, see dlthub job logs and dlthub job runs logs.
From the Web UI
Select any run on the Jobs page to open its run detail page, which provides:
- Status bar — status badge, trigger type, profile, start and end timestamps, and elapsed time (live-updating while the run is in progress)
- Pipeline runs table — every dlt pipeline executed during the job, with row counts and status
- Log viewer — real-time streaming logs (refreshed each second while the run is active) or static logs once a run has completed
Understand run states
| Status | Meaning |
|---|---|
| Pending | Run is queued, waiting to start |
| Starting | Run is being initialized |
| Running | Actively executing |
| Completed | Finished without errors |
| Failed | Encountered an error — check logs for details |
| Cancelled | Manually stopped via CLI or Web UI |
Diagnose a failed run
From the CLI:
dlthub job runs info <name> [run#]— check exit status and timingdlthub job logs <name> [run#]— read the error output
From the Web UI:
- Inspect the logs — the log viewer on the run detail page contains the full execution output, including stack traces.
- Review the pipeline runs — the pipeline-runs table on the run detail page lists each dlt pipeline executed during the job and its outcome. Open an individual pipeline run for detailed load information (tables loaded, row counts, bytes, duration).
- Consult the dashboard — the Dashboard and Pipelines pages surface success-rate trends that help identify recurring issues.
- Verify the deployment — the Deployment & Config page indicates the currently deployed code version. Sync the latest changes with
dlthub deploy.
Common causes of failures:
- Missing dependencies in
pyproject.toml— all packages must be declared, not just locally installed (rundlthub local run <job>first to catch this) - Secrets not configured for the
prodprofile — the platform usesprodfor batch jobs; check.dlt/prod.secrets.toml - Script missing
if __name__ == "__main__":— the job does nothing without it dev_mode=Trueleft in — drops and recreates the dataset on every run, destroying production data- Wrong destination credentials — the
prodprofile may point to a different destination thandev - Job timeout — default is 120 minutes; override with
execute={"timeout": "6h"}(see Execution constraints)
Cancel an active run
Cancel the latest active run of a job:
dlthub job runs cancel my_pipeline.py
Cancel a specific run number:
dlthub job runs cancel my_pipeline.py 5
Cancel active runs across multiple matching jobs (for example everything tagged ingest):
dlthub job cancel "tag:ingest"
# preview without cancelling
dlthub job cancel "tag:ingest" --dry-run
See dlthub job runs cancel and dlthub job cancel. Cancellation is also available from the run detail page and the Jobs page context menu.
Monitor pipeline metrics
The Web UI at app.dlthub.com provides built-in dashboards and visualizations of pipeline telemetry, removing the need for an external observability stack. The Pipelines page surfaces aggregated metrics per pipeline:
- Success rate — percentage of successful runs over time
- Rows loaded — total data volume trends
- Duration — performance trends for identifying regressions
- Charts — time-series visualizations with toggleable views (Runs, Rows, Bytes, Duration)
The Dashboard page provides a workspace-wide overview, summarizing job status and recent run activity.
Pipeline dashboard in the platform UI
The same pipeline and dataset troubleshooting dashboard you run locally is available directly in the dltHub platform Web UI under the Notebooks section. Open it from there to inspect schemas, query loaded data, and review run traces for pipelines deployed on the platform — no local setup required.
Known limitations
Batch jobs have a configurable maximum runtime. Jobs exceeding this limit are automatically cancelled. See Platform limits for details.
Troubleshooting
No 'access' profile detected
Your interactive notebooks will use the prod (or default) configuration. Create access.config.toml and access.secrets.toml with read-only credentials.
No 'prod' profile detected
Batch jobs will use the default configuration. Create prod.config.toml and prod.secrets.toml with read/write credentials.
Job not using latest code
The CLI does not yet detect whether local code differs from remote. Run dlthub workspace deployment sync (or any run / serve / deploy) to ensure your latest code is deployed.
Logs not appearing
Logs typically lag a few seconds during execution and are guaranteed complete after the run finishes. Wait for the run to complete, or stream them in real time:
dlthub job logs my_pipeline.py --follow