apoc.periodic.iterate yields corrupted data

[michaelncy]

Expected Behavior (Mandatory)

We're using an apoc.periodic.iterate query to delete relationships. Normally it yields all the output parameters correctly (for example updateStatistics is a map with the relevant fields).

Actual Behavior (Mandatory)

We see where there's a deadlock in the query, the data that is yielded from the query (even though it succeeds after the deadlock) is corrupted, for example the updateStatistics is an int instead of a map.
When trying to yield only specific output parameters, the problem disappears it seems.

How to Reproduce the Problem

Run an apoc.periodic.iterate query that deletes relationships with yielding all the output parameters.
In the same time run a query that creates relationships from one of the nodes in the first query to trigger a deadlock.

Simple Dataset (where it's possibile)

// First Query
CALL apoc.periodic.iterate("
    MATCH (:TEST)<-[r:REL]-(:BLA {a: $a}) RETURN r",
    "DELETE r",
    {batchSize: 1000, parallel: false, retries: 5,  params: {a: $a}})

// Second Query - Pass a list of records to simulate a query that updates a batch of data
UNWIND $records AS record
MERGE (a:TEST)-[b:REL2]->(c:TEST2 {b: record})
ON CREATE SET a.test = "test"

Steps (Mandatory)

1. Run both queries at the same time to ensure a deadlock happens using the python client
2. Check the output from the first query

Screenshots (where it's possibile)

Specifications (Mandatory)

Currently used versions

Versions

• OS: Neo4j Aura 2025.07
• Neo4j-Apoc: 2025.07.1
• Client: neo4j-python/5.26.0 Python-Rust/3.12.7-final-0 (linux)

[loveleif]

Hello @michaelncy , thanks for taking the time to report and sorry for the inconvenience.

In apoc.periodic.iterate it's not allowed to pass nodes or relationships between the "iterate" query and the "action" query (see Rebinding Nodes and Relationships). The query in your example becomes:

CALL apoc.periodic.iterate("
    MATCH (:TEST)<-[r:REL]-(:BLA {a: $a}) RETURN id(r) AS id",
    "MATCH ()-[r]->() WHERE id(r) = id DELETE r",
    {batchSize: 1000, parallel: false, retries: 5,  params: {a: $a}})

[loveleif]

Also note that CALL subqueries in transactions can replace most of what apoc.periodic.iterate do using plain Cypher. Your example could look something like this:

MATCH (:TEST)<-[r:REL]-(:BLA {a: $a})
CALL {
  DELETE r
} IN TRANSACTIONS 
    OF 1000 ROWS 
    ON ERROR RETRY FOR 30 SECONDS

[michaelncy]

Hello @michaelncy , thanks for taking the time to report and sorry for the inconvenience.

In apoc.periodic.iterate it's not allowed to pass nodes or relationships between the "iterate" query and the "action" query (see Rebinding Nodes and Relationships). The query in your example becomes:

CALL apoc.periodic.iterate("
    MATCH (:TEST)<-[r:REL]-(:BLA {a: $a}) RETURN id(r) AS id",
    "MATCH ()-[r]->() WHERE id(r) = id DELETE r",
    {batchSize: 1000, parallel: false, retries: 5,  params: {a: $a}})

Thanks for the clarification! We'll modify our query and see if the issue resolves.
Regarding "call in transactions", we're using managed transactions, and from what we understood, it can't be used with managed transactions. Is it correct?
Thanks!

[loveleif]

Regarding "call in transactions", we're using managed transactions, and from what we understood, it can't be used with managed transactions. Is it correct?

That is correct @michaelncy. Two of the reasons for this is that CALL subqueries in transactions can't be rolled back (which managed transactions allows you to do) or retried (driver will retry failed queries for you in some circumstances). Even though there is nothing stopping you from using managed transactions with apoc.periodic.iterate, it's also problematic. For example consider this scenario (pseudo code):

tx = driver.beginTx()
tx.run("CALL apoc.periodic.iterate('UNWIND range(1, 100000) AS i', 'CREATE (:Node {i:i})', {})")
tx.rollback()

Even with the rollback, all the 100 000 nodes are committed.

[michaelncy]

Hello @michaelncy , thanks for taking the time to report and sorry for the inconvenience.

In apoc.periodic.iterate it's not allowed to pass nodes or relationships between the "iterate" query and the "action" query (see Rebinding Nodes and Relationships). The query in your example becomes:

CALL apoc.periodic.iterate("
MATCH (:TEST)<-[r:REL]-(:BLA {a: $a}) RETURN id(r) AS id",
"MATCH ()-[r]->() WHERE id(r) = id DELETE r",
{batchSize: 1000, parallel: false, retries: 5, params: {a: $a}})

Hi @loveleif I've tried to run the same scenario with the modified query provided and the issue still reproduced. We've also tried to run on our dev environment with "yield only specific output parameters" as I've mentioned in the ticket in order to avoid this issue, but we've still encountered the same problem, although seems like it's happening less frequently. Another error I saw on our dev env along side the original issue was: "ValueError('Invalid string length marker: 111')" originating from:

  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/work/result.py", line 605, in single
    await self._buffer(2)
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/work/result.py", line 443, in _buffer
    async for record in self:
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/work/result.py", line 393, in __aiter__
    await self._connection.fetch_message()
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/io/_common.py", line 195, in inner
    await coroutine_func(*args, **kwargs)
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/io/_bolt.py", line 991, in fetch_message
    tag, fields = await self.inbox.pop(
                  ^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_async/io/_common.py", line 81, in pop
    self._unpacker.unpack(hydration_hooks) for _ in range(size)
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/appuser/apps/test/.venv/lib/python3.12/site-packages/neo4j/_codec/packstream/v1/__init__.py", line 290, in unpack
    value, i = _rust_unpack(
               ^^^^^^^^^^^^^
ValueError: Invalid string length marker: 111

[michaelncy]

We've tried switching back to the regular neo4j python driver (Without the Rust extension), and the issue doesn't seem to reproduce, opening a ticket for them as well

[loveleif]

Thanks, I'll do more investigation.

[michaelncy]

Thanks, I'll do more investigation.

Actually it was a bug in the rust driver extension, which is already fixed in the latest version, I'm closing this issue, thanks for the help!

[loveleif]

Actually it was a bug in the rust driver extension, which is already fixed in the latest version, I'm closing this issue, thanks for the help!

@michaelncy thanks for letting me know!