Skip to content

fix(logging): create the log file on the first record - #5167

Merged
paul-nechifor merged 5 commits into
mainfrom
paul/fix/no-empty-cli-log-files
Nov 11, 2026
Merged

paul-nechifor merged 5 commits into
mainfrom
paul/fix/no-empty-cli-log-files

Conversation

@paul-nechifor

@paul-nechifor paul-nechifor commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Why do we need this?

Every dimos CLI call left a logs/dimos_<timestamp>_<pid>.jsonl file behind, even when the command logged nothing. Commands like dimos status or dimos topic echo each added an empty file. One test checkout had about 55,000 of these files, which makes ls logs useless.

dimos run had a smaller version of the same problem. It logs "Starting DimOS" before the run directory exists, so each run left a one-line file next to its logs/<run-id>/ directory.

What is this feature?

The log file handler opened its file as soon as a logger was created, which happens at import time. It now opens the file when the first record is written. A process that logs nothing leaves no file.

When dimos run switches to the per-run directory, the records it already wrote are moved into the run's main.jsonl and the per-process file is removed. dimos log now shows those early records too.

Both places that built the file handler now share one helper.

@greptile-apps

greptile-apps Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR changes file creation and routing across DimOS logging, including run startup.

The handler cleanup requirement must be satisfied before merging.

Findings

  1. P2 Tests leave loggers changed ▶

Summary

This PR opens log files only when the first record is written and moves early records into the run's main.jsonl.

  • Log files now appear only after a logger writes a record.
  • Run logs now include records written before the run folder exists.

Reviews (1) · Last reviewed commit: "fix(logging): create the log file on the..." · Reviewed by Greptile

Comment thread dimos/utils/test_logging_config.py Resolved

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

go2 replay realtime (arm64)

Details
Benchmark suite Current: 60c2c4d Previous: 1916f35 Ratio
peak memory 1733.07 MB 1740.68 MB 1.00
peak threads 374 threads 377 threads 0.99
network (transport) 2733.739 MB 2730.983 MB 1.00
disk write 1.852 MB 1.586 MB 1.17
instructions 137.219 G 138.613 G 0.99

This comment was automatically generated by workflow using github-action-benchmark.

@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 94.00000% with 3 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
dimos/utils/logging_config.py 87.50% 1 Missing and 2 partials ⚠️
@@           Coverage Diff           @@
##             main    #5167   +/-   ##
=======================================
  Coverage   83.74%   83.75%           
=======================================
  Files        1866     1866           
  Lines      182158   182200   +42     
  Branches    15008    15012    +4     
=======================================
+ Hits       152549   152594   +45     
+ Misses      25569    25565    -4     
- Partials     4040     4041    +1     
Components Coverage Δ
Tests 96.66% <100.00%> (-0.01%) ⬇️
Flag Coverage Δ
OS-macos-latest 78.01% <86.00%> (+<0.01%) ⬆️
OS-ubuntu-24.04-arm 77.78% <86.00%> (-0.01%) ⬇️
OS-ubuntu-latest 78.14% <86.00%> (+<0.01%) ⬆️
Py-3.10 78.14% <86.00%> (-0.01%) ⬇️
Py-3.11 78.12% <86.00%> (+<0.01%) ⬆️
Py-3.12 78.13% <86.00%> (+<0.01%) ⬆️
SelfHosted-Linux 40.76% <68.00%> (+<0.01%) ⬆️
SelfHosted-macOS 40.49% <70.00%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
dimos/core/test_per_run_logs.py 100.00% <100.00%> (ø)
dimos/utils/test_logging_config.py 100.00% <100.00%> (ø)
dimos/utils/logging_config.py 83.63% <87.50%> (+2.42%) ⬆️

... and 3 files with indirect coverage changes

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Nov 10, 2026
@paul-nechifor
paul-nechifor enabled auto-merge (squash) November 11, 2026 04:07
@github-actions github-actions Bot removed the ready-to-merge Required CI checks have passed on this PR label Nov 11, 2026
Every process that imported a DimOS logger opened logs/dimos_<timestamp>_<pid>.jsonl
at import time, so short CLI commands left an empty file behind. The file handler
now opens the file lazily.

dimos run also logged a few records before the run directory existed. Those
records now move into the run's main.jsonl.
test_forked_child_cannot_move_parent_startup_log fails on every run with
--cov. The CI re-run step uses --no-cov, so the failure looked flaky.

conftest starts coverage in every test subprocess. The test's script
runs in tmp_path, where "dimos" in source is a package name and not a
directory. Coverage starts a second instance in the forked child, and
that instance counts the package as measured only when dimos/__init__.py
runs. A fork never imports it again, so the child prints the warning on
stderr, which the test requires to be empty.
@paul-nechifor
paul-nechifor force-pushed the paul/fix/no-empty-cli-log-files branch from 8021e3d to 60c2c4d Compare November 11, 2026 10:29
@github-actions github-actions Bot added the ready-to-merge Required CI checks have passed on this PR label Nov 11, 2026
@paul-nechifor
paul-nechifor merged commit 8b0c375 into main Nov 11, 2026
55 checks passed
@paul-nechifor
paul-nechifor deleted the paul/fix/no-empty-cli-log-files branch November 11, 2026 10:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Required CI checks have passed on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants