Skip to content

Add configuration options to duration extension #13469

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
120dd48
Add n_durations config option
user27182 Apr 5, 2025
235b52d
Add write_durations option
user27182 Apr 5, 2025
a5a73b6
Add options and update docs
user27182 Apr 6, 2025
221a25c
Update changes.rst
user27182 Apr 6, 2025
72bfb24
Fix ruff errors
user27182 Apr 6, 2025
23294e8
Fix ruff errors
user27182 Apr 6, 2025
c85a9e2
Fix lint errors
user27182 Apr 6, 2025
84d7fc1
Ruff formatting
user27182 Apr 6, 2025
96f5a65
noqa timezone
user27182 Apr 6, 2025
ba8cbb8
Add tests
user27182 Apr 6, 2025
42d53ac
Add tests
user27182 Apr 6, 2025
04bf8b8
Modify testroots
user27182 Apr 6, 2025
920bbc4
Rename write_durations -> write_json
user27182 Apr 10, 2025
d1bf297
Update n_slowest tests
user27182 Apr 10, 2025
6c5e824
Make options top-level
user27182 Apr 10, 2025
e4b27b8
Update docs and formatting
user27182 Apr 11, 2025
8d754a1
Add option to specify json path
user27182 Apr 11, 2025
25675a5
Fix internal default
user27182 Apr 11, 2025
7cbd32e
Change test root
user27182 Apr 11, 2025
8b90a4e
Merge branch 'master' into feat/durations_options
user27182 Apr 11, 2025
346c729
Fix typing
user27182 Apr 11, 2025
0bf3216
Change test root
user27182 Apr 11, 2025
a7d4ec5
Fix typing
user27182 Apr 11, 2025
9de5011
Fix typing
user27182 Apr 11, 2025
1631065
Change test root
user27182 Apr 11, 2025
1965046
Update docs
user27182 Apr 11, 2025
2131604
Fix ref
user27182 Apr 11, 2025
5cb3637
Add extensions cross-ref
user27182 Apr 11, 2025
61ab2ec
Update docs
user27182 Apr 11, 2025
e1b9f77
Fix line too long
user27182 Apr 11, 2025
85ff9d5
Add duration_limit config option
user27182 Apr 13, 2025
2ec2aa8
Fix typing
user27182 Apr 13, 2025
2dbc395
Use freshenv
user27182 Apr 13, 2025
df33ed3
Version added
user27182 Apr 13, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS.rst
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ Contributors
* Eric Larson -- better error messages
* Eric N. Vander Weele -- autodoc improvements
* Eric Wieser -- autodoc improvements
* Erik Bedard -- config options for :mod:`sphinx.ext.duration`
* Etienne Desautels -- apidoc module
* Ezio Melotti -- collapsible sidebar JavaScript
* Filip Vavera -- napoleon todo directive
Expand Down
1 change: 1 addition & 0 deletions CHANGES.rst
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ Features added
* #13439: linkcheck: Permit warning on every redirect with
``linkcheck_allowed_redirects = {}``.
Patch by Adam Turner.
* #13468: Add config options to :mod:`sphinx.ext.duration`.

Bugs fixed
----------
Expand Down
4 changes: 4 additions & 0 deletions doc/usage/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1416,6 +1416,7 @@ Options for warning control
* ``autosectionlabel.<document name>``
* ``autosummary``
* ``autosummary.import_cycle``
* ``duration``
* ``intersphinx.external``

You can choose from these types. You can also give only the first
Expand Down Expand Up @@ -1481,6 +1482,9 @@ Options for warning control
``ref.any``,
``toc.duplicate_entry``, ``toc.empty_glob``, and ``toc.not_included``.

.. versionadded:: 8.3
``duration``.


Builder options
===============
Expand Down
92 changes: 89 additions & 3 deletions doc/usage/extensions/duration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,92 @@

.. versionadded:: 2.4

This extension measures durations of Sphinx processing and show its
result at end of the build. It is useful for inspecting what document
is slowly built.
This extension measures durations of Sphinx processing when reading
documents and is useful for inspecting what document is slowly built.
Durations are printed to console at the end of the build and saved
to a JSON file in :attr:`~sphinx.application.Sphinx.outdir` by default.

Enable this extension by adding it to your :confval:`extensions`
configuration.

.. code-block:: python

extensions = [
...
'sphinx.ext.duration',
]

Configuration
=============

.. confval:: duration_print_slowest
:type: :code-py:`bool`
:default: :code-py:`True`

Show the slowest durations in the build summary. The durations
are sorted in order from slow to fast. This prints up to
:confval:`duration_n_slowest` durations to the console, e.g.:

.. code-block:: shell

====================== slowest 5 reading durations =======================
0.012s toctree
0.011s admonitions
0.011s refs
0.006s docfields
0.005s figure

.. versionadded:: 8.3

.. confval:: duration_n_slowest
:type: :code-py:`int`
:default: :code-py:`5`

Maximum number of slowest durations to show in the build summary
when :confval:`duration_print_slowest` is enabled. Only the ``5``
slowest durations are shown by default. Set this to ``0`` to show
all durations.

.. versionadded:: 8.3

.. confval:: duration_print_total
:type: :code-py:`bool`
:default: :code-py:`True`

Show the total reading duration in the build summary, e.g.:

.. code-block:: shell

====================== total reading duration ==========================
Total time reading 31 files:

minutes: 0
seconds: 3
milliseconds: 142

.. versionadded:: 8.3

.. confval:: duration_write_json
:type: :code-py:`str | bool`
:default: :code-py:`'sphinx_reading_durations.json'`

Write all reading durations to a JSON file in
:attr:`~sphinx.application.Sphinx.outdir`. The file contents are
dict-like and contain the document file paths (relative to
:attr:`~sphinx.application.Sphinx.outdir`) as keys and reading
durations in seconds as values. Set this value to an empty
string or ``False`` to disable writing the file, or set it to a
relative path to customize it.

This may be useful for testing and setting a limit on reading times.

.. versionadded:: 8.3

.. confval:: duration_limit
:type: :code-py:`float | None`
:default: :code-py:`None`

Set a duration limit (in seconds) for reading a document. If any
duration exceeds this value, a warning is emitted.

.. versionadded:: 8.3
114 changes: 103 additions & 11 deletions sphinx/ext/duration.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,13 @@

from __future__ import annotations

import json
import time
from datetime import datetime, timedelta, timezone
from itertools import islice
from operator import itemgetter
from typing import TYPE_CHECKING
from pathlib import Path
from typing import TYPE_CHECKING, cast

import sphinx
from sphinx.domains import Domain
Expand All @@ -24,6 +27,14 @@ class _DurationDomainData(TypedDict):
reading_durations: dict[str, float]


DEFAULT_OPTIONS = {
'duration_print_total': True,
'duration_print_slowest': True,
'duration_n_slowest': 5,
'duration_write_json': 'sphinx_reading_durations.json',
'duration_limit': None,
}

logger = logging.getLogger(__name__)


Expand All @@ -39,6 +50,15 @@ def reading_durations(self) -> dict[str, float]:
def note_reading_duration(self, duration: float) -> None:
self.reading_durations[self.env.docname] = duration

def warn_reading_duration(self, duration: float, duration_limit: float) -> None:
logger.warning(
__('Reading duration %s exceeded the duration limit %s'),
_format_seconds(duration),
_format_seconds(duration_limit),
type='duration',
location=self.env.docname,
)

def clear(self) -> None:
self.reading_durations.clear()

Expand Down Expand Up @@ -75,22 +95,79 @@ def on_doctree_read(app: Sphinx, doctree: nodes.document) -> None:
domain = app.env.domains['duration']
domain.note_reading_duration(duration)

duration_limit = cast(
'float | None',
getattr(app.config, 'duration_limit', DEFAULT_OPTIONS['duration_limit']),
)
if duration_limit is not None and duration > duration_limit:
domain.warn_reading_duration(duration, duration_limit)


def on_build_finished(app: Sphinx, error: Exception) -> None:
"""Display duration ranking on the current build."""
domain = app.env.domains['duration']
if not domain.reading_durations:
reading_durations = domain.reading_durations
if not reading_durations:
return
durations = sorted(
domain.reading_durations.items(), key=itemgetter(1), reverse=True
)

logger.info('')
logger.info(
__('====================== slowest reading durations =======================')
)
for docname, d in islice(durations, 5):
logger.info(f'{d:.3f} {docname}') # NoQA: G004
# Get default options and update with user-specified values
options = DEFAULT_OPTIONS.copy()
for key in options:
options[key] = getattr(app.config, key, DEFAULT_OPTIONS[key])

if options['duration_print_total']:
logger.info('')
logger.info(
__(
'====================== total reading duration =========================='
)
)

n_files = len(reading_durations)
s = '' if n_files == 1 else 's'

total = sum(reading_durations.values())
logger.info('Total time reading %d file%s:\n', n_files, s)
logger.info(_format_seconds(total, multiline=True))

if options['duration_print_slowest']:
sorted_durations = sorted(
reading_durations.items(), key=itemgetter(1), reverse=True
)
n_slowest = cast('int', options['duration_n_slowest'])

if n_slowest == 0:
n_slowest = len(sorted_durations)
fmt = ' '
else:
n_slowest = min(n_slowest, len(sorted_durations))
fmt = f' {n_slowest} '

logger.info('')
logger.info(
__(
f'====================== slowest{fmt}reading durations ======================='
)
)

for docname, d in islice(sorted_durations, n_slowest):
logger.info('%s %s', _format_seconds(d), docname)

logger.info(__(''))

if write_json := options['duration_write_json']:
# Write to JSON
relpath = (
Path(write_json)
if isinstance(write_json, (Path, str))
else Path(cast('Path | str', DEFAULT_OPTIONS['duration_write_json']))
)
out_file = Path(app.builder.outdir) / relpath
# Make sure all parent dirs exist
for parent in out_file.parents[: len(relpath.parents) - 1]:
parent.mkdir(exist_ok=True)
with out_file.open('w', encoding='utf-8') as fid:
json.dump(reading_durations, fid, indent=4)


def setup(app: Sphinx) -> dict[str, bool | str]:
Expand All @@ -105,3 +182,18 @@ def setup(app: Sphinx) -> dict[str, bool | str]:
'parallel_read_safe': True,
'parallel_write_safe': True,
}


def _format_seconds(seconds: float, multiline: bool = False) -> str:
"""Convert seconds to a formatted string."""
if not multiline:
return f'{seconds:.3f}s'
dt = datetime(1, 1, 1, tzinfo=timezone.utc) + timedelta(seconds=seconds) # noqa: UP017
minutes = dt.hour * 60 + dt.minute
seconds = dt.second
milliseconds = round(dt.microsecond / 1000.0)
return (
f'minutes: {minutes:>3}\n'
f'seconds: {seconds:>3}\n'
f'milliseconds: {milliseconds:>3}'
)
Loading
Loading