Real analysis example

A finding you can inspect.

A historical TinyDB case, shown the way a Wofanu report presents it.

This case was reported and fixed upstream before we analysed it, and it was used while developing and checking the analysis. It shows what a finding contains. It is not a measure of how often the analysis finds problems.

This report was published under our earlier name, Fidrail. Its content is unchanged.

Historical case · TinyDB #629 / fix #633 · Known fix

Two documents can receive the same ID.

In the studied mixed insertion batch, the later write can replace the earlier document.

tinydb/table.py · Table.insert_multiple

Why does it matter?

The table keeps only the later document. The earlier one is lost without an error, and the returned IDs repeat.

What supports it?

Static analysis

  • Establishedunder these assumptions

    Before the fix, with an empty table and no cached ID, the dict’s ID comes from stored data read before the batch commits (line 743), so it is 1. If the explicit ID is also 1, the write at line 214 replaces the one at line 206.

  • Establishedunder these assumptions

    With the fix, the next ID comes from the IDs already in the batch’s working table, so nothing is replaced.

  • Not established

    Whether any real caller passes a batch like this.

What was tested?

Runtime check · run separately

Before the fix[1, 1]One document lost
With the fix[1, 2]Both kept

What should I do?

Inspect the ID allocator, _get_next_id, and the order of writes in the batch. Then run a mixed-batch regression test, like the one added with the fix.

Open the full evidenceHide the full evidence

Source trace

Code before the fix, 19066e0, numbered in the order it runs. Each step is established under the assumptions listed with this report. Elided lines are marked ⋮.

tinydb/table.py · insert_multipleView at 19066e0
193if isinstance(document, self.document_class):
204 doc_id = document.doc_id
205 doc_ids.append(doc_id)
1206 table[doc_id] = dict(document)
207 continue
2212doc_id = self._get_next_id()
213doc_ids.append(doc_id)
4214table[doc_id] = dict(document)
tinydb/table.py · _get_next_idView at 19066e0
727def _get_next_id(self):
3733 if self._next_id is not None:
734 next_id = self._next_id
735 self._next_id = next_id + 1
736
737 return next_id
3743 table = self._read_table()
  1. 1

    Establishedline 206

    The explicit ID is written into the batch’s working table.

  2. 2

    Establishedline 212

    The next item asks _get_next_id for an ID.

  3. 3

    Establishedlines 733 and 743

    No ID is cached, so the allocator reads stored data. That data doesn’t include step 1’s write yet, so the allocator returns 1.

  4. 4

    Establishedline 214

    If the explicit ID is also 1, this write uses the same key and replaces the document from step 1.

  5. Not establishedcallers

    Whether any real caller passes a batch like this. The trace stops here.

The fix, for comparison

tinydb/table.py · insert_multipleView commit bd034d0
206 table[doc_id] = dict(document)
+207 # Same as insert(): don't reuse this ID for later
+208 # auto-assigned documents in this batch.
+209 self._next_id = None
210 continue
+215next_id = self._next_id
+216if next_id is None:
+217 if table:
+218 next_id = max(
+219 self.document_id_class(i) for i in table.keys()
+220 ) + 1
+221 else:
+222 next_id = 1
+223self._next_id = next_id
224doc_id = self._get_next_id()

The fix clears the cached ID after an explicit ID (line 209). It then works out the next ID from the IDs already in the working table (lines 215–223), before the unchanged allocator call (line 224). Lines marked + were added by the fix. Static analysis of the fix finds no replacement under the same assumptions.

Runtime check

Run separately from the static analysis, on the exact code before and after the fix, with in-memory storage. Returned IDs:

Batch caseBefore the fixWith the fix
Empty table[1, 1][1, 2]
Existing records[6, 6][6, 7]
Cached next ID[2, 2][2, 3]

Before the fix, the explicit document was lost in all three cases, and records already in the table were kept. Controls with automatic IDs only, single inserts and duplicate-ID rejection behaved as expected in both versions. These runs don’t change what the static analysis established.

Assumptions

Case analysed: an empty table with no cached ID, and a batch of an explicit-ID Document followed by a plain dict. The static result holds under these recorded assumptions:

  1. The result is conditional. It doesn’t show that any real caller makes this call.conditional_model_not_actual_public_caller_qualification
  2. The receiver is TinyDB’s own Table, with no subclass override that affects this behavior.declared_owner_receiver_without_affecting_override
  3. The table’s query cache starts empty, as it does by default, and nothing re-enters the table during the batch.empty_default_owned_query_cache_no_reentry
  4. Document IDs are exact built-in integers, stored under their canonical keys.exact_builtin_integer_ids_canonical_stored_keys
  5. The class and its defaults are used as written, with nothing overriding them in between.exact_owner_and_source_defaults_no_intervening_override
  6. The documents are plain mappings without custom hooks.inert_exact_mapping_payloads_no_custom_hooks
  7. Attribute lookup and built-ins behave normally and aren’t patched.ordinary_unpatched_lookup_and_builtins
  8. The batch is a complete list or tuple of exactly two items that doesn’t change during the call.stable_complete_two_item_list_or_tuple
  9. Storage reads before the commit are stable, and the batch writes to storage only at its normal commit.stable_precommit_external_reads_and_confined_normal_write_boundary
  10. No code outside the analysed source changes this state during the call.static_source_model_no_unseen_external_mutation

What remains uncertain

  • Not established

    Whether any real caller passes a batch like this.

  • Not established

    Whether the committed result is durable in storage.

  • Not analysed

    Cases outside this one: a table with existing records, a cached next ID, longer batches and custom storage. The runtime check covers the first two; the static result doesn’t.

Record

Code
Before the fix, 19066e0; the fix, bd034d0 (PR #633). Release v4.9.0 has the same insert_multiple code.
Analysis
Mapper 11ce47c, the analysis engine behind Fidrail.
Assessment
Both static results matched the retained runtime check. The case was known in advance, so the assessment wasn’t blinded.
Upstream
Issue #629, with the reporter’s reproduction on Python 3.11.

This case was reported and fixed upstream before we analysed it, and it was used while developing and checking the analysis. It shows what a finding contains. It is not a measure of how often the analysis finds problems.

How to read the labels

Evidence

  • EstablishedSupported by the analysis under the stated assumptions.
  • CandidateA possible relation that needs confirmation.
  • Not establishedThe analysis couldn’t settle it. Shown as a gap.
  • DisprovedThe analysis shows the relation doesn’t hold.

Coverage and outcome

  • Not analysedNot examined: out of scope, unavailable to the analysis or not attempted. Nothing is concluded about it.
  • Rule violationBreaks a rule you declared, with adequate evidence.
  • Known fixA historical case already fixed upstream.

How to read the labels

Reports answer three separate questions, each with its own labels.

Evidence

  • EstablishedSupported by the analysis under the stated assumptions.
  • CandidateA possible relation that needs confirmation.
  • Not establishedThe analysis couldn’t settle it. Shown as a gap.
  • DisprovedThe analysis shows the relation doesn’t hold.

Coverage and outcome

  • Not analysedNot examined: out of scope, unavailable to the analysis or not attempted. Nothing is concluded about it.
  • Rule violationBreaks a rule you declared, with adequate evidence.
  • Known fixA historical case already fixed upstream.