JSON

The JSON component is an operator component that allows users to manipulate and convert JSON entities. It can carry out the following tasks:

#Release Stage

Alpha

#Configuration

The component definition and tasks are defined in the definition.json and tasks.json files respectively.

#Supported Tasks

#Marshal

Convert JSON to a string

InputIDTypeDescription
Task ID (required)taskstringTASK_MARSHAL
JSON (required)jsonanyJSON entity to be marshaled. It can be any valid JSON datatype (e.g. number, string, hash, array).
OutputIDTypeDescription
JSON StringstringstringString representation of the JSON input.

#Unmarshal

Convert a string to JSON

InputIDTypeDescription
Task ID (required)taskstringTASK_UNMARSHAL
String (required)stringstringJSON string to be unmarshaled. It can represent any valid JSON datatype (e.g. number, string, hash, array).
OutputIDTypeDescription
JSONjsonanyJSON entity extracted from the string input.

#jq

Process JSON through a jq command

InputIDTypeDescription
Task ID (required)taskstringTASK_JQ
JSON Valuejson-valueanyJSON entity to be processed by the filter. It can be any valid JSON datatype (e.g. number, string, hash, array).
Filter (required)jq-filterstringFilter, in jq syntax, that will be applied to the JSON input.
OutputIDTypeDescription
ResultsresultsarrayThe jq command results.

jq defines a syntax to "transform JSON in various ways, by selecting, iterating, reducing and otherwise mangling JSON documents". Depending on the command input and the jq filter, the type and shape of the results may vary.

Here are some examples on how the jq syntax works.

Input JSONjq filterOutput
{"foo": 128}.foo[128]
{"a": {"b": 42}}.a.b[42]
{"id": "sample", "10": {"b": 42}}{(.id): .["10"].b}[{ "sample": 42 }]
[{"id":1},{"id":2},{"id":3}].[] | .id[1, 2, 3]
{"a":1,"b":2}.a += 1 | .b *= 2[{ "a": 2, "b": 4 }]
{"a":1} [2] 3. as {$a} ?// [$a] ?// $a | $a[1, 2, 3]

There's a common pitfall in jq, which doesn't support the dash (-) character in dictionary key names and interprets them as a subtraction. The way to access such fields in a jq filter is by wrapping the key in double quotes:

Input JSONjq filterOutput
[{"key-a": "value1"}, {"key-a": "value2"}].[] | ."key-a"["value1", "value2"]

#Rename Fields

Rename fields in a JSON object with a conflict resolution strategy. If the new field already exists, it will be overwritten by default.

InputIDTypeDescription
Task ID (required)taskstringTASK_RENAME_FIELDS
JSON (required)jsonanyJSON object to be edited. It can be any valid JSON datatype (e.g. number, string, hash, array).
Fields (required)fieldsarray[object]An array of objects specifying the fields to be renamed.
Conflict Resolutionconflict-resolutionstringDefines how conflicts are handled when the new field already exists in the data. If the new field already exists, it will be overwritten by default. If the new field does not exist, it will be created. When set to 'error', the component will throw an error if the new field already exists. When set to 'skip', the new field will be skipped if it already exists.
Enum values
  • overwrite
  • skip
  • error
Input Objects in Rename Fields

Fields

An array of objects specifying the fields to be renamed.

FieldField IDTypeNote
FromfromstringThe field in the original data that needs to be renamed.
TotostringThe new name for the field.
OutputIDTypeDescription
JSONjsonanyJSON object with the renamed fields.

#Example Recipes

Recipe for the Resume Screening pipeline.


version: v1beta
component:
json-0:
type: json
task: TASK_UNMARSHAL
input:
string: ${openai-0.output.texts[0]}
openai-0:
type: openai
task: TASK_TEXT_GENERATION
input:
model: gpt-4o-2024-08-06
n: 1
prompt: |
Given an ${variable.resume} and a ${pdf-to-text.output.body}, create an automated system to screen and convert this information into a structured candidate profile. The system should extract key information such as:
Skills: Identify and list relevant skills mentioned in the resume.
Experience: Extract work history including job titles, companies, durations, and responsibilities from the resume.
Education: Capture educational background including degrees, institutions, and graduation dates from the resume.
Certifications: Identify any professional certifications or additional qualifications in the resume.
Fit Score: Calculate a fit score based on the alignment of the candidate's profile with the job description, taking into account required skills, experience level, and education.
response-format:
json-schema: |-
{
"name": "resume_response",
"strict": true,
"schema": {
"type": "object",
"properties": {
"name": {
"type": "string"
},
"education": {
"type": "string"
},
"score": {
"type": "number"
},
"reasoning": {
"type": "string"
},
"experience": {
"type": "array",
"items": {
"type": "string"
}
},
"skills": {
"type": "array",
"items": {
"type": "string"
}
}
},
"required": [
"name",
"education",
"score",
"reasoning",
"experience",
"skills"
],
"additionalProperties": false
}
}
type: json_schema
system-message: You are a helpful assistant.
temperature: 1
top-p: 1
setup:
api-key: ${secret.INSTILL_SECRET}
pdf-to-text:
type: document
task: TASK_CONVERT_TO_TEXT
input:
document: ${variable.resume}
variable:
job-description:
title: job description
description: The text of the job description.
format: string
resume:
title: resume
description: The PDF file of the candidates resume
format: "*/*"
output:
output:
title: output
value: ${json-0.output.json}