Configuration

The needs-config-writer extension provides several configuration options to control how Sphinx-Needs configuration is exported to ubproject.toml.

All configuration options are set in your Sphinx conf.py file.

needscfg_outpath

Type: str

Default: "${outdir}/ubproject.toml"

Specifies the output path where the ubproject.toml file will be written.

The path supports template variables:

• ${outdir} - Replaced with the Sphinx output directory (build directory)

• ${srcdir} - Replaced with the Sphinx source directory

Relative paths are interpreted relative to the configuration directory (where conf.py is located).

Examples:

# Write to build output directory (default)
needscfg_outpath = "${outdir}/ubproject.toml"

# Write to source directory
needscfg_outpath = "${srcdir}/ubproject.toml"

# Write to custom subdirectory in output
needscfg_outpath = "${outdir}/config/needs.toml"

# Relative path (relative to conf.py location)
needscfg_outpath = "generated/ubproject.toml"

# Absolute path
needscfg_outpath = "/absolute/path/to/ubproject.toml"

needscfg_warn_on_diff

Type: bool

Default: False

Controls whether to emit a warning when the existing output file content differs from the new configuration being generated.

When enabled and the output file already exists:

• The extension compares the existing file content with the new content

• If they differ, emits a warning (subtype: content_diff)

• Whether the file is updated depends on needscfg_overwrite

Behavior:

• False (default): No warning is emitted when content changes

• True: Emits a warning when existing file content differs from new configuration

Examples:

# No warning on content changes (default)
needscfg_warn_on_diff = False

# Warn when configuration changes
needscfg_warn_on_diff = True

Tip

Enable this in CI/CD pipelines to detect unexpected configuration changes.

needscfg_overwrite

Type: bool

Default: True

Controls whether to overwrite an existing output file when the configuration content differs.

Behavior:

• True (default): Overwrites the file when content differs

• False: Does not overwrite the file when content differs (logs info message instead)

Examples:

# Automatically update when configuration changes (default)
needscfg_overwrite = True

# Prevent overwriting existing files
needscfg_overwrite = False

Note

When needscfg_overwrite is False and content differs, the extension will log an info message but not update the file. This is useful to prevent accidentally overwriting manually edited configuration files.

needscfg_write_all

Type: bool

Default: False

Controls whether to include all Sphinx-Needs configuration values (including defaults) or only explicitly configured values.

Behavior:

• False (default): Only writes configuration values that were explicitly set in conf.py

• True: Writes all Sphinx-Needs configuration values, including default values

Examples:

# Write only explicitly configured values (default)
needscfg_write_all = False

# Write all configuration including defaults
needscfg_write_all = True

Tip

Set this to True if you want to see the complete configuration with all defaults, useful for documentation or when migrating configuration to ubproject.toml.

needscfg_add_header

Type: bool

Default: True

Controls whether to add an auto-generated warning header to the output file.

When enabled, the extension adds a comment header at the top of the generated TOML file warning that the file is auto-generated and should not be manually modified.

Behavior:

• True (default): Adds header comment to the output file

• False: Generates output file without the header comment

Examples:

# Add auto-generated warning header (default)
needscfg_add_header = True

# Skip the header comment
needscfg_add_header = False

The generated header looks like this:

# This file is auto-generated by needs-config-writer.
# Do not manually modify it - changes will be overwritten.

Tip

Keep this enabled (True) to remind users that the file is auto-generated and manual changes will be lost on the next build.

needscfg_exclude_vars

Type: list[str]

Default: ["needs_from_toml", "needs_from_toml_table", "needs_schema_definitions_from_json"]

Specifies which Sphinx-Needs configuration variables should be excluded from writing to the output file. This is typically used to exclude resolved configuration values that should not be duplicated in the output.

The default list excludes:

• needs_from_toml - Configuration loaded from TOML files

• needs_from_toml_table - Parsed TOML table data

• needs_schema_definitions_from_json - Schema definitions loaded from JSON

Behavior:

• Variable names in this list will be filtered out

• Only affects variables that start with needs_

• The check is performed on the full attribute name (e.g., needs_from_toml)

Examples:

# Use default exclusions (recommended)
needscfg_exclude_vars = [
    "needs_from_toml",
    "needs_from_toml_table",
    "needs_schema_definitions_from_json",
]

# Add custom exclusions
needscfg_exclude_vars = [
    "needs_from_toml",
    "needs_from_toml_table",
    "needs_schema_definitions_from_json",
    "needs_custom_variable",  # Exclude a custom variable
]

# Exclude only specific variables
needscfg_exclude_vars = ["needs_from_toml"]

# No exclusions (not recommended - may cause duplicates)
needscfg_exclude_vars = []

Warning

Removing the default exclusions may cause resolved configuration values to be written to the output file, potentially creating circular dependencies or duplicate configurations.

needscfg_merge_toml_files

Type: list[str]

Default: [] (empty list)

Specifies a list of TOML file paths to shallow-merge into the final output configuration. This allows you to include additional configuration from external TOML files into the generated ubproject.toml.

The paths support the same template variables as needscfg_outpath:

• ${outdir} - Replaced with the Sphinx output directory (build directory)

• ${srcdir} - Replaced with the Sphinx source directory

Relative paths are interpreted relative to the configuration directory (where conf.py is located).

Merge behavior:

• Files are processed in the order they appear in the list

• Each file is shallow-merged (top-level keys only) into the configuration

• If a TOML file has a [needs] table, only that table is merged

• If no [needs] table exists, the entire file content is merged

• Values from merged files override values from the Sphinx configuration

• Later files in the list override earlier files

Use cases:

• Add project-specific metadata not available in Sphinx config

• Include version information from separate TOML files

• Merge team-wide configuration standards

• Add deployment-specific settings

Examples:

# Merge a single additional configuration file
needscfg_merge_toml_files = ["additional_config.toml"]

# Merge multiple files (processed in order)
needscfg_merge_toml_files = [
    "${srcdir}/team_defaults.toml",
    "project_overrides.toml",
]

# Use build output directory
needscfg_merge_toml_files = ["${outdir}/generated_metadata.toml"]

Example TOML file with needs table:

# additional_config.toml
[needs]
project_version = "1.2.3"
build_date = "2025-10-28"

Example TOML file without needs table:

# additional_config.toml
project_version = "1.2.3"
build_date = "2025-10-28"

Both formats work - if a [needs] table exists, only its contents are merged.

Note

If a merge file doesn’t exist, a warning is emitted but the build continues. Failed file loads (e.g., invalid TOML syntax) also emit warnings without stopping the build.

Tip

Use this feature to separate dynamic configuration (like version numbers or build metadata) from static Sphinx-Needs configuration in conf.py.

needscfg_exclude_defaults

Type: bool

Default: False

Controls whether to exclude configuration options that are set to their default values.

When enabled, the extension compares each Sphinx-Needs configuration value with its default value. If they match, the option is excluded from the output file.

Behavior:

• False (default): Include all configuration values (subject to needscfg_write_all and needscfg_exclude_vars)

• True: Exclude configuration values that match their default values

Use cases:

• Generate cleaner configuration files with only explicitly set values

• Reduce noise in version-controlled configuration files

• Make it easier to see what’s been customized vs. defaults

• Minimize file size for generated configuration

Examples:

# Include all values, even defaults (default behavior)
needscfg_exclude_defaults = False

# Exclude values that match defaults
needscfg_exclude_defaults = True
needscfg_write_all = True  # Usually combined with write_all

Note

This option works in combination with needscfg_write_all:

• When needscfg_write_all = False: Only explicitly set values are included (default behavior)

• When needscfg_write_all = True and needscfg_exclude_defaults = False: All values including defaults are included

• When needscfg_write_all = True and needscfg_exclude_defaults = True: All values are considered but defaults are filtered out

Tip

Enable both needscfg_write_all and needscfg_exclude_defaults to generate configuration that includes all customized values while excluding unchanged defaults. This provides a clean view of what’s been explicitly configured.

needscfg_relative_path_fields

Type: list[str | dict]

Default: []

Specifies which configuration fields should have their absolute paths converted to relative paths in the output file. This is particularly useful when working with build systems like Bazel that generate absolute paths, but you want the configuration file to use relative paths for portability.

Each entry in the list can be either:

1. String format (simple field pattern):

   • "needs_schema_debug_path" - Matches the field directly

   • "needs_external_needs[*].json" - Supports * wildcards for array indices

2. Dict format (for paths embedded in strings with prefix/suffix):

   • field (required): The field pattern to match (e.g., "needs_flow_configs.my_config")

   • prefix (optional): String prefix before the path (e.g., "!include ")

   • suffix (optional): String suffix after the path (e.g., "?raw=true")

When a field matches a pattern:

1. Check if the value is an absolute path (Path object or string)

2. Extract the path portion (removing prefix/suffix if configured)

3. Calculate a relative path from the output file location to the target path

4. Replace with relative path (preserving prefix/suffix if configured)

Configuration Formats:

needscfg_relative_path_fields = [
    # Simple string format - for direct path values
    "needs_schema_debug_path",

    # String with wildcards - for array fields
    "needs_external_needs[*].json_path",

    # Dict with prefix - for paths embedded in strings like "!include /path/to/file"
    {
        "field": "needs_flow_configs.plantuml_config",
        "prefix": "!include ",
    },

    # Dict with suffix - for paths like "/path/to/file?option=value"
    {
        "field": "needs_asset_url",
        "suffix": "?raw=true",
    },

    # Dict with both prefix and suffix
    {
        "field": "needs_custom_path",
        "prefix": "file://",
        "suffix": "#anchor",
    },

    # Dict with just field (equivalent to string format)
    {
        "field": "needs_build_json_path",
    },
]

Use Cases:

• Working with Bazel or similar build systems that use absolute paths

• Making configuration files portable across different machines/environments

• Handling PlantUML !include directives with absolute paths

• Processing URL-like strings with path components

• Keeping paths relative to the repository root instead of absolute system paths

Example with Bazel:

If you have a Bazel-generated path like:

# In conf.py (generated by Bazel)
needs_schema_debug_path = "/home/user/.cache/bazel/.../execroot/_main/bazel-out/k8-fastbuild/bin/docs.runfiles/project/schema_debug"

And your configuration file output is at:

/home/user/git/project/docs/ubproject.toml
/home/user/git/project/bazel-out > /home/user/.cache/bazel/.../execroot/_main/bazel-out

Note that Bazel creates a bazel-out symlink in the project directory (/home/user/git/project/bazel-out) that points into the Bazel cache. The extension detects this symlink and uses it to create a shorter relative path.

With this setting:

needscfg_relative_path_fields = ["needs_schema_debug_path"]

The output will contain:

[needs]
schema_debug_path = "../bazel-out/k8-fastbuild/bin/docs.runfiles/project/schema_debug"

Example with Prefix (PlantUML !include directive):

If you have a PlantUML configuration with an !include directive:

# In conf.py
needs_flow_configs = {
    "plantuml_theme": "!include /home/user/project/assets/theme.puml"
}

With this configuration:

needscfg_relative_path_fields = [
    {
        "field": "needs_flow_configs.plantuml_theme",
        "prefix": "!include ",
    }
]

The output will preserve the !include prefix with a relative path:

[needs.flow_configs]
plantuml_theme = "!include ../assets/theme.puml"

Example with Suffix (URL parameters):

For paths that include URL parameters or anchors:

# In conf.py
needs_asset_url = "/home/user/project/docs/image.png?width=500"

needscfg_relative_path_fields = [
    {
        "field": "needs_asset_url",
        "suffix": "?width=500",
    }
]

The output preserves the suffix:

[needs]
asset_url = "../image.png?width=500"

Note

• All relative paths are converted to POSIX format (forward slashes) on all platforms, including Windows. This ensures configuration files are portable across operating systems.

• The extension correctly handles output file paths that don’t exist yet (common for generated configuration files) by detecting file suffixes like .toml or .json.

• On Unix systems, the extension attempts to find symlinks (like Bazel’s bazel-out) to create shorter relative paths when possible.

• If no common ancestor exists (e.g., paths on different drives on Windows), the absolute path will be returned unchanged.

Examples

Minimal setup

# conf.py
extensions = [
    "sphinx_needs",
    "needs_config_writer",
]

This will write the configuration to ${outdir}/ubproject.toml, updating it whenever the configuration changes. The file contents can be manually copied over to a new primary ubproject.toml to migrate existing conf.py configuration.

Development setup

# conf.py
extensions = [
    "sphinx_needs",
    "needs_config_writer",
]

needscfg_outpath = "ubproject.toml"
needscfg_overwrite = True
needscfg_warn_on_diff = False

This configuration writes the file to the directory holding the conf.py file, useful during development to keep configuration in version control. Allow overwriting as the original is version controlled. Any diffs will show up.

Full configuration export

# conf.py
extensions = [
    "sphinx_needs",
    "needs_config_writer",
]

needscfg_write_all = True
needscfg_outpath = "${outdir}/full_config.toml"

This exports the complete configuration including all defaults.

CI/CD setup

# conf.py
extensions = [
    "sphinx_needs",
    "needs_config_writer",
]

needscfg_warn_on_diff = True
needscfg_overwrite = False
needscfg_outpath = "ubproject.toml"

This configuration emits warnings when configuration changes and prevents overwriting, allowing you to catch unexpected configuration drift in CI/CD pipelines.