org.wikimedia.eventutilities.flink.formats.json

Class JsonRowDeserializationSchema

java.lang.Object

org.wikimedia.eventutilities.flink.formats.json.JsonRowDeserializationSchema

All Implemented Interfaces:

Serializable, org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.types.Row>, org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

public class JsonRowDeserializationSchema
extends Object
implements org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.types.Row>

Deserialization schema from JSON to Flink Row type. NOTE: This class was directly copied and modified from Flink 1.15.0's deprecated org.apache.flink.formats.json.JsonRowSerializationSchema. It is used to bypass the Table API when you want a DataStream of Row without needing to initialize a TableEnvironment. WMF changes from upstream Flink: - Deserialize into name based Rows instead of position based Rows. - Supports ISO8601 date times with timezone offsets (+00:00), upstream does only support trailing 'Z'

Deserializes a byte[] message as a JSON object and reads the specified fields.

Failures during deserialization are forwarded as wrapped IOExceptions. TODO: - Should we return a Row in name based mode? - Should we infer the RowKind automatically? https://phabricator.wikimedia.org/T310082

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JsonRowDeserializationSchema.Builder Builder for JsonRowDeserializationSchema.
static class	JsonRowDeserializationSchema.RowFieldMode Used to determine which of the Row field modes in which this deserializer should create Rows.

Nested classes/interfaces inherited from interface org.apache.flink.api.common.serialization.DeserializationSchema

org.apache.flink.api.common.serialization.DeserializationSchema.InitializationContext

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	JsonRowDeserializationSchema(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo, boolean failOnMissingField, boolean ignoreParseErrors) Default constructor that will deserialize Rows in hybrid POSITION_WITH_NAMES mode.
protected	JsonRowDeserializationSchema(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo, boolean failOnMissingField, boolean ignoreParseErrors, JsonRowDeserializationSchema.RowFieldMode rowFieldMode)

Method Summary

Modifier and Type	Method and Description
org.apache.flink.types.Row	convert(com.fasterxml.jackson.databind.JsonNode root) Convert a JsonNode to a Row following this schema.
org.apache.flink.types.Row	deserialize(byte[] message)
boolean	equals(Object o)
org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row>	getProducedType()
int	hashCode()
boolean	isEndOfStream(org.apache.flink.types.Row nextElement)

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.api.common.serialization.DeserializationSchema

deserialize, open

Constructor Detail

JsonRowDeserializationSchema

protected JsonRowDeserializationSchema(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo,
                                       boolean failOnMissingField,
                                       boolean ignoreParseErrors,
                                       JsonRowDeserializationSchema.RowFieldMode rowFieldMode)

JsonRowDeserializationSchema

protected JsonRowDeserializationSchema(org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> typeInfo,
                                       boolean failOnMissingField,
                                       boolean ignoreParseErrors)

Default constructor that will deserialize Rows in hybrid POSITION_WITH_NAMES mode.

Method Detail

deserialize

public org.apache.flink.types.Row deserialize(byte[] message)
                                       throws IOException

Specified by:

deserialize in interface org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.types.Row>

Throws:

IOException

convert

public org.apache.flink.types.Row convert(com.fasterxml.jackson.databind.JsonNode root)

Convert a JsonNode to a Row following this schema.

isEndOfStream

public boolean isEndOfStream(org.apache.flink.types.Row nextElement)

Specified by:

isEndOfStream in interface org.apache.flink.api.common.serialization.DeserializationSchema<org.apache.flink.types.Row>

getProducedType

public org.apache.flink.api.common.typeinfo.TypeInformation<org.apache.flink.types.Row> getProducedType()

Specified by:

getProducedType in interface org.apache.flink.api.java.typeutils.ResultTypeQueryable<org.apache.flink.types.Row>

equals

public boolean equals(Object o)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object