Expose simdjson via experimental flag or node:simdjson module

[pmbanugo]

I'm proposing for Node.js to expose the existing [simdjson]((https://github.com/simdjson/simdjson) integration through an experimental flag or via node:simdjson to enable those who want to use it without brower compatibility concern.

Background

Node.js currently includes simdjson internally but doesn't expose it to userland code AFAIU. During a recent discussion with @RafaelGSS on his livestream, it was mentioned that simdjson isn't enabled yet primarily due to insufficient testing to demonstrate API compatibility with browser JSON APIs.

Motivation

• Performance: simdjson offers significant parsing performance improvements over the standard JSON parser
• Community Testing: Exposing it experimentally would allow the community to extensively test compatibility
• Feedback Loop: Early access would help identify compatibility issues and API design concerns
• Innovation: Enables developers to experiment with high-performance JSON parsing in Node.js applications

Proposed Solution

Expose simdjson through one of these approaches:

Option 1: Experimental Flag

node --experimental-simdjson app.js

This would make simdjson available globally, potentially replacing or augmenting the standard JSON object.

Option 2: Dedicated Module

import { parse, stringify } from 'node:simdjson';
// or
const { parse, stringify } = require('node:simdjson');

Option 3: Hybrid Approach

Enable both the experimental flag and module access, allowing for different testing scenarios.

API Considerations

The exposed API may aim for compatibility (if desired) with:

• Standard JSON.parse() and JSON.stringify() signatures
• Browser JSON API behavior and edge cases
• Error handling patterns consistent with existing Node.js APIs

Implementation Considerations

• Mark the API as experimental with appropriate warnings
• Document known limitations or differences from standard JSON
• Provide clear migration path for when/if the API becomes stable
• Consider feature detection mechanisms for applications

---

I'd love to try out simdjson in my apps instead of the V8 JSON functions.

An experimental exposure could provide valuable data on real-world compatibility and performance, helping to make an informed decision about full integration while giving the community access to simdjson's performance benefits.

[legendecas]

V8 has integrated SIMD into the built-in JSON APIs: https://v8.dev/blog/json-stringify.

I think re-invent the full JSON api on top of simdjson (as mentioned in the motivation, covering all edge cases and error handling) is re-implementing what V8 is doing.

#58491 should bring in the V8 changes into Node.js.

[RafaelGSS]

During a recent discussion with @RafaelGSS on his livestream,

Right after closing the live stream, I saw the recent V8 update on JSON.stringify 😄. However, I believe it would be great to monitor it. If we expose a kind of util.stringify() using simdjson implementation in a draft PR, we could investigate the performance comparison between these two implementations.

[lemire]

@legendecas

V8 has integrated SIMD into the built-in JSON APIs: https://v8.dev/blog/json-stringify.

The stringify function generates JSON. It is not related to parsing.

There are several Node.js parsers built on simdjson, such as https://github.com/luizperes/simdjson_nodejs (just search for 'simdjson' on npm).

[joyeecheung]

From what I can tell, if the API tries to interact with JS objects in V8 in whole, using simdjson is not likely to yield much difference, or it's likely to be slower, because object creation (for parse) / iteration (stringify) via the C++ API can become the bottleneck. In Node.js internally, simdjson is only yielding a difference because the callsite are in C++ and they are selectively reading specific fields and passing those values into JS in a simplified format (e.g. an array with hard-coded indexes of select fields), but not serializing the entire object into JS. To yield a performance difference, the API needs to be selective (e.g. just querying the value of a specific path).

[pmbanugo]

I think that clarifies it for me @joyeecheung. Thank you.