Best Way to Read Json File in Java

If yous're working in a statically-typed language like Java then dealing with JSON can be tricky. JSON doesn't have type definitions and is lacking some features which we would like - at that place'southward simply strings, numbers, booleans and null, so to store other types (similar dates or times) nosotros're forced to use a string-based convention. Despite its shortcomings, JSON is the most common format for APIs on the spider web and so nosotros need a way to piece of work with it in Coffee.

Jackson is one of the most popular Java JSON libraries, and is the ane I use most frequently. In this post I'll pick a fairly complex JSON document and three queries which I want to make using Jackson. I'll compare iii dissimilar approaches:

1. Tree model
2. Information binding
3. Path queries

All the lawmaking used in this post is in this repository. It'll work with Coffee 8 onwards.

Other Java Libraries for working with JSON

The well-nigh pop Java libraries for working with JSON, equally measured by usage in maven central and GitHub stars, are Jackson and Gson. In this post I will be using Jackson, and there is an equivalent post with Gson lawmaking examples here.

You can see the Jackson dependency for the examples here.

Example data and questions

To find some example data I read Tilde'due south recent mail 7 cool APIs you didn't know y'all needed, and picked out the About Earth Object Spider web Service API from the NASA APIs. This API is maintained by the brilliantly-named SpaceRocks squad.

The NeoWS Feed API request returns a list of all asteroids whose closest approach to World is inside the next 7 days. I'll be showing how answer the following questions in Coffee:

• How many are at that place?
  This can exist found by looking at the element_count fundamental at the root of the JSON object.
• How many of them are potentially hazardous?
  We demand to loop through each NEO and bank check the is_potentially_hazardous_asteroid key, which is a boolean value in the JSON. (Spoiler: information technology's non zippo 😨)
• What is the proper name and speed of the fastest Nearly Earth Object?
  Over again, we demand to loop through simply this time the objects are more complex. Nosotros as well demand to be aware that speeds are stored as strings not numbers, eg "kilometers_per_second": "6.076659807" . This is common in JSON documents equally it avoids precision problems on very minor or very large numbers.

A tree model for JSON

Jackson allows you to read JSON into a tree model: Java objects that stand for JSON objects, arrays and values. These objects are called things like JsonNode or JsonArray and are provided by Jackson.

Pros:

• Y'all will not need to create any extra classes of your own
• Jackson can practise some implicit and explicit type coercions for you

Cons:

• Your lawmaking that works with Jackson's tree model objects can be verbose
• It's very tempting to mix Jackson code with application logic which tin can make reading and testing your lawmaking hard

Jackson tree model examples

Jackson uses a class called ObjectMapper as its primary entrypoint. Typically I create a new ObjectMapper at application startup and considering ObjectMapper instances are thread-safe it's OK to care for it like a singleton.

An ObjectMapper tin can read JSON from a variety of sources using an overloaded readTree method. In this case I provided a String. readTree returns a JsonNode which represents the root of the JSON document. JsonNode instances tin be JsonObjects, JsonArrays or a variety of "value" nodes such as TextNode or IntNode.

Here is the code to parse a String of JSON into a JsonNode:

            ObjectMapper mapper = new ObjectMapper(); JsonNode neoJsonNode = mapper.readTree(SourceData.asString());                      

[this code in the case repo]

How many NEOs are in that location?

We need to observe the element_count fundamental in the JsonNode, and return it as an int. The code reads quite naturally:

            int getNeoCount(JsonNode neoJsonNode) {    return neoJsonNode        .become("element_count")        .asInt(); }                      

Error handling: if element_count is missing then .get("element_count") will return zilch and there will be a NullPointerException at .asInt(). Jackson can apply the Nothing Object Pattern to avoid this kind of exception, if you apply .path instead of .get. Either way yous will accept to handle errors, so I tend to use .get.

How many potentially hazardous asteroids are at that place this calendar week?

I admit that I expected the answer hither to be zero. It's currently 19 - but I'm not panicking (yet). To calculate this from the root JsonNode we need to:

• iterate through all the NEOs - there is a list of these for each date so nosotros volition need a nested loop
• increment a counter if the is_potentially_hazardous_asteroid field is true

Here'south the lawmaking:

            int getPotentiallyHazardousAsteroidCount(JsonNode neoJsonNode) {    int potentiallyHazardousAsteroidCount = 0;    JsonNode nearEarthObjects = neoJsonNode.path("near_earth_objects");    for (JsonNode neoClosestApproachDate : nearEarthObjects) {        for (JsonNode neo : neoClosestApproachDate) {            if (neo.get("is_potentially_hazardous_asteroid").asBoolean()) {                potentiallyHazardousAsteroidCount += 1;            }        }    }    return potentiallyHazardousAsteroidCount; }                      

[this code in the example repo]

This is a little awkward - Jackson does not straight allow us to use the Streams API so I'one thousand using nested for loops. asBoolean returns the value for boolean fields in the JSON only can also exist called on other types:

• numeric nodes will resolve as true if nonzero
• text nodes are true if the value is "true" .

What is the name and speed of the fastest NEO?

The method of finding and iterating through each NEO is the aforementioned equally the previous case, but each NEO has the speed nested a few levels deep so you lot demand to descend through to pick out the kilometers_per_second value.

            "close_approach_data": [  {     ...      "relative_velocity": {          "kilometers_per_second": "6.076659807",          "kilometers_per_hour": "21875.9753053124",          "miles_per_hour": "13592.8803223482"    },   ...  } ]                      

I created a pocket-sized grade to concord both values called NeoNameAndSpeed. This could exist a record in the hereafter. The lawmaking creates one of those objects like this:

            NeoNameAndSpeed getFastestNEO(JsonNode neoJsonNode) {    NeoNameAndSpeed fastestNEO = null;    JsonNode nearEarthObjects = neoJsonNode.path("near_earth_objects");    for (JsonNode neoClosestApproachDate : nearEarthObjects) {        for (JsonNode neo : neoClosestApproachDate) {            double speed = neo                .get("close_approach_data")                .go(0)                .get("relative_velocity")                .get("kilometers_per_second")                .asDouble();            if ( fastestNEO == naught ||  speed > fastestNEO.speed ){                fastestNEO = new NeoNameAndSpeed(neo.get("proper name").asText(), speed);            }        }    }    return fastestNEO; }                      

[this code in the case repo]

Even though the speeds are stored as strings in the JSON I could telephone call .asDouble() - Jackson is smart plenty to call Double.parseDouble for me.

Data binding JSON to custom classes

If you lot accept more complex queries of your data, or you need to create objects from JSON that you can pass to other code, the tree model isn't a good fit. Jackson offers some other mode of performance called information bounden, where JSON is parsed directly into objects of your design. By default Spring MVC uses Jackson in this way when you lot accept or return objects from your web controllers.

Pros:

• JSON to object conversion is straightforward
• reading values out of the objects can utilize whatever Java API
• the objects are independent of Jackson and then can be used in other contexts
• the mapping is customizable using Jackson Modules

Cons:

• Upward-front work: yous have to create classes whose structure matches the JSON objects, then have Jackson read your JSON into these objects.

Data bounden 101

Hither'southward a simple example based off a pocket-size subset of the NEO JSON:

            {  "id": "54016476",  "name": "(2020 GR1)",  "closeApproachDate": "2020-04-12", }                      

We could imagine a form for belongings that data like this:

            course NeoSummaryDetails {     public int id;     public String name;     public LocalDate closeApproachDate; }                      

Jackson is almost able to map dorsum and forth between JSON and matching objects like this out of the box. It copes fine with the int id actually beingness a string, only needs some assist converting the String 2020-04-12 to a LocalDate object. This is done with a custom module, which defines a mapping from JSON to custom object types.

Jackson information bounden - custom types

For the LocalDate mapping Jackson provides a dependency. Add this to your projection and configure your ObjectMapper like this:

                          ObjectMapper objectMapper = new ObjectMapper()        .registerModule(new JavaTimeModule());                      

[this code in the example repo]

Jackson data binding - custom field names

You might have noticed that I used closeApproachDate in my case JSON to a higher place, where the data from NASA has close_approach_date. I did that considering Jackson volition use Java's reflection capabilities to match JSON keys to Java field names, and they need to lucifer exactly.

Nearly times you can't change your JSON - it'south usually coming from an API that you lot don't command - but yous still wouldn't like to have fields in your Java classes written in snake_case. This could take been done with an annotation on the closeApproachDate field:

            @JsonProperty("close_approach_date") public LocalDate closeApproachDate;                      

[this code in the case repo]

Creating your custom objects with JsonSchema2Pojo

Correct now yous are probably thinking that this can go very time-consuming. Field renaming, custom readers and writers, not to mention the sheer number of classes you might need to create. Well, you lot're correct! Only fear not, there's a not bad tool to create the classes for you.

JsonSchema2Pojo tin can take a JSON schema or (more usefully) a JSON document and generate matching classes for you lot. It knows about Jackson annotations, and has tons of options, although the defaults are sensible. Unremarkably I find it does 90% of the piece of work for me, but the classes often need some finessing once they're generated.

To use it for this project I removed all but one of the NEOs and selected the post-obit options:

[generated code in the example repo]

Data stored in keys and values

The NeoWS JSON has a slightly bad-mannered (but not unusual) feature - some data is stored in the keys rather than the values of the JSON objects. The near_earth_objects map has keys which are dates. This is a chip of a problem because the dates won't always be the same, but of course jsonschema2pojo doesn't know this. Information technology created a field called _20200412. To ready this I renamed the class _20200412 to NeoDetails and the blazon of nearEarthObjects became Map<Cord, List<NeoDetails>> (see that here). I could then delete the now-unused NearEarthObjects grade.

I as well changed the types of numbers-in-strings from String to double and added LocalDate where appropriate.

Jackson data binding for the Near-Globe Object API

With the classes generated by JsonSchema2Pojo the whole large JSON document tin can be read with:

            NeoWsDataJackson neoWsDataJackson = new ObjectMapper()    .registerModule(new JavaTimeModule())    .readValue(SourceData.asString(), NeoWsDataJackson.class);                      

Finding the data we desire

Now that we take plain old Java objects we tin can use field admission and the Streams API to observe the data we want:

            System.out.println("NEO count: " + neoWsData.elementCount);   System.out.println("Potentially hazardous asteroids: " +    neoWsData.nearEarthObjects.values()        .stream().flatMap(Collection::stream) // this converts a Collection of Collections of objects into a single stream        .filter(neo -> neo.isPotentiallyHazardousAsteroid)        .count());   NeoDetails fastestNeo = neoWsData.nearEarthObjects.values()    .stream().flatMap(Collection::stream)    .max( Comparator.comparing( neo -> neo.closeApproachData.get(0).relativeVelocity.kilometersPerSecond ))    .get();  System.out.println(String.format("Fastest NEO is: %due south at %f km/sec",    fastestNeo.proper noun,    fastestNeo.closeApproachData.get(0).relativeVelocity.kilometersPerSecond));                      

[this code in the example repo]

This code is more natural Java, and it doesn't have Jackson all through it then it would be easier to unit examination this version. If you are working with the same format of JSON a lot, the investment of creating classes is probable to be well worth it.

Path queries from JSON using JsonPointer

With Jackson you lot can also use a JSON Pointer which is a compact way to refer to a specific single value in a JSON document:

            JsonNode node = objectMapper.readTree(SourceData.asString());  JsonPointer pointer = JsonPointer.compile("/element_count");  System.out.println("NEO count: " + node.at(pointer).asText());                      

[this code in the example repo]

JSON Pointers can only indicate to a single value - y'all can't aggregate or use wildcards, so they are rather limited.

Summing up the dissimilar ways of using Jackson

For unproblematic queries, the tree model can serve you well but you volition most likely be mixing up JSON parsing and application logic which tin brand information technology hard to examination and maintain.

To pull out a unmarried value from a JSON certificate, y'all might consider using a JSON pointer, but the lawmaking is barely any simpler than using the tree model and so I never do.

For more complex queries, and especially when your JSON parsing is part of a larger application, I recommend data bounden. Information technology's usually easiest in the long run, remembering that JsonSchema2Pojo tin practise most of the work for yous.

What are your favourite ways of working with JSON in Java? Let me know on Twitter I'm @MaximumGilliard, or past email I am mgilliard@twilio.com. I tin can't wait to see what {"you": "build"} .

lampeliser1956.blogspot.com

Source: https://www.twilio.com/blog/java-json-with-jackson

Share this post