How can I incrementally update a global JSON graph for NER across multiple text chunks?

I am performing named entity recognition on first-person biographies to output a JSON-formatted network graph of people and their relationships. My prompt instructs the model:

"Perform named entity recognition on the text. Only extract the entities that are people. Give each person a unique id. Save any biographical information like name, place of birth, occupation, interests, and personality traits. Find the pair-wise relationships between these people. Structure the data as a network graph where the nodes are people, and the edges are their pairwise relationships, with the name of the relationship as the label property and the identified perceived feelings listed in an array in a "sentiments" property of the relation. Output the results in JSON format."

This works generally, but if I hit the maximum context length, the JSON output sometimes remains incomplete.

I considered processing the text in batches by splitting it into chunks. However, challenges arise when context is split across chunks. For example, if a biography mentions two people named Sarah—one being the speaker’s mother and the other the daughter named after her—the model usually disambiguates correctly when given the full context. When processing in chunks, the model must retain earlier context to resolve these ambiguities.

Example: From the first chunk, a sentence like:

"My mother Sarah was born in 1943. I named my daughter after her."

produces a partial graph:

JSONCopied
{
  nodes: [
    {
      id: 'speaker',
      type: 'person',
      name: 'Michael'
    },
    {
      id: 'sarah1',
      type: 'person',
      name: 'Sarah'
    },
    {
      id: 'sarah2',
      type: 'person',
      name: 'Sean'
    }
  ],
  edges: [
    {
      source: 'speaker',
      target: 'sarah2',
      type: 'relationship',
      relationship: 'Mother'
    },
    {
      source: 'sarah1',
      target: 'speaker',
      type: 'relationship',
      relationship: 'Mother'
    }
  ]
}

For the second chunk, consider the sentence:

"Sarah’s husband, John, and my mother don’t get along very well."

Which, on its own, might yield:

JSONCopied
{
  nodes: [
    {
      id: 'speaker',
      type: 'person',
      name: 'Michael'
    },
    {
      id: 'sarah',
      type: 'person',
      name: 'Sarah'
    },
    {
      id: 'mother',
      type: 'person',
      name: ''
    },
    {
      id: 'john',
      type: 'person',
      name: 'John'
    }
  ],
  edges: [
    {
      source: 'mother',
      target: 'speaker',
      type: 'relationship',
      relationship: 'Mother'
    },
    {
      source: 'sarah',
      target: 'john',
      type: 'relationship',
      relationship: 'Married'
    }
  ]
}

The question is: How can I process chunk 2 and subsequent chunks so that the model retains previous context and incrementally updates the growing JSON graph? Specifically, how do I include the output from each chunk in the prompt so the model merges new NER results with the existing data structure?

One effective approach suggested in the discussion is to maintain a global state by including the current JSON graph at the beginning of each new prompt, along with clear instructions to merge or update the graph with new entities and relationships. For example, the prompt might look like:

PLAINTEXTCopied
You have the following JSON graph built from previous text:
{
  "nodes": [
    {"id": "speaker", "type": "person", "name": "Michael"},
    {"id": "sarah1",  "type": "person", "name": "Sarah"},
    {"id": "sarah2",  "type": "person", "name": "Sarah"}
  ],
  "edges": [
    {"source": "speaker", "target": "sarah2", "type": "relationship", "relationship": "Mother"},
    {"source": "sarah1", "target": "speaker", "type": "relationship", "relationship": "Mother"}
  ]
}

Now process the following new chunk of text:
"Sarah’s husband, John, and my mother don’t get along very well."

Update the JSON graph to add any new entities and relationships, merging entities already seen. Output the complete updated graph in valid JSON.

This approach, often called "incremental prompting," helps ensure the model has all prior context to resolve ambiguities (like the two different Sarahs) and correctly merge the data. What strategies or improvements would you recommend for managing such context across multiple chunks?