Jsonify — XML to JSON

pygixml.jsonify serializes XML directly to JSON. “Directly” is the operative word: the in-memory entry points (dumps() and friends) traverse the pugixml DOM in C++ and write straight into a JSON string buffer — no intermediate Python dict/list is ever built, unlike going through pygixml.dictify.parse() followed by json.dumps(). The streaming entry points (stream_dump(), stream_jsonl()) go a step further and skip even the DOM, converting a giant XML file to a giant JSON file in roughly constant memory.

The output shape matches pygixml.dictify.parse() exactly (same @-prefixed attributes, #text for mixed content, repeated siblings collapsed into arrays) — jsonify.dumps(xml) is equivalent to, but faster than, json.dumps(dictify.parse(xml)).

from pygixml import jsonify

xml = """
<database name="users_db">
    <user id="101">
        <name>Mohammad</name>
        <balance>450.75</balance>
    </user>
    <tag>active</tag>
    <tag>verified</tag>
</database>
"""

jsonify.dumps(xml)
# '{"database": {"@name": "users_db", "user": {"@id": "101", ...}, "tag": ["active", "verified"]}}'

jsonify.dumps(xml, pretty=True, indent="  ")   # multi-line, 2-space indent

In-memory entry points

pygixml.jsonify.dumps(source, attr_prefix='@', cdata_key='#text', force_list=None, pretty=False, indent='\\t', encoding='utf-8')

Smart dispatcher — serializes XML to JSON regardless of what form the XML is already in:

Parameters:

  • source – XML string, an already-parsed ObjectifiedElement, or an XMLNode.

  • attr_prefix (str) – Prefix prepended to attribute keys. Default "@".

  • cdata_key (str) – Key used for text content in mixed nodes. Default "#text".

  • force_list (set or True or None) – Tag names that should always be wrapped in a JSON array, even when only one sibling exists. Pass True to force every tag.

  • pretty (bool) – Indent the output. Default False (compact).

  • indent (str) – Indentation string used when pretty=True. Default a tab.

Returns:

JSON string.

Return type:

str

Raises:

  • PygiXMLError – If the XML is malformed.

  • TypeError – If source’s type isn’t recognized.

  • ValueError – If source is a str that doesn’t look like XML (file paths are rejected here on purpose — use dumps_file() explicitly for files).

Note

File input is intentionally excluded from the dispatcher — call dumps_file() directly for a path, so it’s always unambiguous whether a str argument is XML content or a file path.

pygixml.jsonify.dumps_str(xml, attr_prefix='@', cdata_key='#text', force_list=None, pretty=False, indent='\\t', encoding='utf-8')

Parse an XML string and serialize it directly to JSON.

Parameters:

xml (str) – XML source text.

Returns:

JSON string.

Return type:

str

Raises:

PygiXMLError – If the XML is malformed.

pygixml.jsonify.dumps_file(path, attr_prefix='@', cdata_key='#text', force_list=None, pretty=False, indent='\\t', encoding='utf-8')

Parse an XML file and serialize it directly to JSON, returning the result as a str. For files too big to hold as a JSON string in memory, see stream_dump() instead.

Parameters:

path (str) – Filesystem path to the XML file.

Returns:

JSON string.

Return type:

str

Raises:

PygiXMLError – If the file cannot be read or the XML is malformed.

pygixml.jsonify.dumps_obj(elem, attr_prefix='@', cdata_key='#text', force_list=None, pretty=False, indent='\\t', encoding='utf-8')

Serialize an already-parsed ObjectifiedElement subtree directly to JSON, without re-parsing or re-traversing as a dict first.

Parameters:

elem (pygixml.ObjectifiedElement) – Element to serialize.

Returns:

JSON string.

Return type:

str

Raises:

TypeError – If elem is not an ObjectifiedElement.

from pygixml import objectify, jsonify

root = objectify.from_string(xml)
jsonify.dumps_obj(root.user)            # just the <user> subtree
pygixml.jsonify.dumps_node(node, attr_prefix='@', cdata_key='#text', force_list=None, pretty=False, indent='\\t', encoding='utf-8')

Serialize a low-level XMLNode directly to JSON.

Parameters:

node (pygixml.XMLNode) – Node to serialize.

Returns:

JSON string.

Return type:

str

Raises:

TypeError – If node is not an XMLNode.

Streaming entry points: constant memory, files in and out

The functions above all hold the result (and, except for dumps_obj/dumps_node, the parsed DOM too) in memory — fine for documents that fit comfortably in RAM. For documents that don’t, jsonify has two streaming converters that go file-to-file, entirely in C++, with no pugixml DOM, no Python dict/list/str for individual elements, and no json module anywhere in the call chain.

pygixml.jsonify.stream_dump(xml_path, json_path, attr_prefix='@', cdata_key='#text', force_list=None, indent=0, stack_size=4096, io_buf_size=65536)

Convert a (potentially gigantic) XML file into a single, standard, valid JSON file — in roughly constant memory. Produces exactly what dumps_file() would produce (one JSON value mirroring the whole document, loadable with a plain json.load), just without ever holding the document, or the output, fully in memory.

Parameters:

  • xml_path (str) – Path to the input XML file.

  • json_path (str) – Path to the output JSON file. Overwritten if it exists.

  • attr_prefix (str) – Prefix for XML attribute names in JSON keys. Default "@".

  • cdata_key (str) – JSON key used for text content mixed with attributes or child elements. Default "#text".

  • force_list (set or True or None) – Tag names always serialized as a JSON array. True forces every tag. Default None (a tag becomes an array only once a second sibling with that name actually appears).

  • indent (int) – Spaces per nesting level, same convention as json.dump(..., indent=N). 0 (default) is compact; any positive value pretty-prints.

  • stack_size (int) – Size in bytes of yxml’s internal name stack.

  • io_buf_size (int) – Bytes read per XML I/O operation. Default 64 KB.

Returns:

Number of XML elements processed (informational).

Return type:

int

Raises:

PygiXMLError – On malformed XML, or if the input/output file cannot be opened.

How it stays constant-memory while still producing valid JSON syntax. A JSON array needs to know, before its closing ], whether more items follow — but the parser only finds that out when (and if) a second same-tag sibling actually shows up. Rather than buffer whole subtrees to be safe, the engine writes optimistically and patches the output file in place once it learns more:

  • The first time a child tag is seen under some parent, one placeholder byte is reserved right before its value, and the tag is written as a plain (non-array) value.

  • A second sibling with the same tag arrives → that placeholder byte is overwritten with [ (an O(1) patch), and the new value is appended right after the first. This is the common case for record-oriented XML (same-tag siblings adjacent in the source) and never moves a single byte.

  • A different child tag is interleaved between two same-tag siblings → the engine splices: it shifts just the interleaved bytes forward (in small fixed-size chunks) to open a gap for the new sibling. Cost is proportional to how much was interleaved, not to the file size — and it’s the only case where any data movement happens at all.

from pygixml import jsonify
import json

jsonify.stream_dump("huge.xml", "huge.json")            # compact
jsonify.stream_dump("huge.xml", "huge.json", indent=2)  # pretty

with open("huge.json") as f:
    data = json.load(f)   # a single, ordinary, valid JSON document
pygixml.jsonify.stream_jsonl(xml_path, jsonl_path, tag, attr_prefix='@', cdata_key='#text', force_list=None, stack_size=4096, io_buf_size=65536)

The file-to-file counterpart of iterjsonl() (and the per-tag, filtering complement to stream_dump(), which always converts the entire document): streams an XML file straight to a .jsonl file, one matched element per line, entirely in C++. Unlike iterjsonl(), no StreamElement and no Python str/dict/list is ever created for the matched elements themselves — each element’s JSON object is assembled in a small in-memory buffer (bounded by that one element’s own subtree, the same constant-memory model as iterfind()) and written straight to the file.

Parameters:

  • xml_path (str) – Path to the source XML file.

  • jsonl_path (str) – Path to the .jsonl file to write. Overwritten if it exists.

  • tag (str) – Tag name of the elements to convert and write, one per line.

  • attr_prefix – Same meaning as stream_dump().

  • cdata_key – Same meaning as stream_dump().

  • force_list – Same meaning as stream_dump().

  • stack_size (int) – Size in bytes of yxml’s internal name stack.

  • io_buf_size (int) – Bytes read per XML I/O operation. Default 64 KB.

Returns:

Number of matched elements written.

Return type:

int

Raises:

PygiXMLError – On malformed XML, or if the input/output file cannot be opened.

Note

If tag appears nested inside an already-matched element, that inner occurrence is folded into the outer match as an ordinary nested field (under its own tag-name key) rather than written as a second, separate line — only the outermost occurrence of a match starts a new JSONL record. This only matters for genuinely self-nested tags; a flat list of repeated sibling records (the common case) is unaffected.

from pygixml import jsonify

n = jsonify.stream_jsonl("huge.xml", "huge.jsonl", "record")
print(f"wrote {n} records")

import json
with open("huge.jsonl") as f:
    for line in f:
        record = json.loads(line)   # each line is independent, valid JSON

Choosing the right entry point

You have / want

In memory

Streamed (constant memory)

Whole document → one JSON value

dumps() / dumps_file()

stream_dump()

One record per line, output as a .jsonl file

Loop + write yourself (see iterjsonl())

stream_jsonl()

One record per line, kept as Python str objects

iterjsonl()

(inherently produces a Python object per line — see Streaming — Constant-Memory Parsing for Big XML for the all-C++ alternative when that’s not needed)

Already-parsed ObjectifiedElement / XMLNode

dumps_obj() / dumps_node()

(n/a — already in memory)

See Streaming — Constant-Memory Parsing for Big XML for iterjsonl and the rest of the underlying constant-memory parsing layer that stream_dump and stream_jsonl are built on top of.