Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DO NOT MERGE] Prototype Entities (otep#264) in Java SDK #6855

Draft
wants to merge 16 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 63 additions & 1 deletion docs/apidiffs/current_vs_latest/opentelemetry-sdk-common.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,64 @@
Comparing source compatibility of opentelemetry-sdk-common-1.44.0-SNAPSHOT.jar against opentelemetry-sdk-common-1.43.0.jar
No changes.
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.detectors.ServiceDetector (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW INTERFACE: io.opentelemetry.sdk.resources.EntityDetector
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW FIELD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.resources.EntityDetector INSTANCE
+++ NEW METHOD: PUBLIC(+) java.util.List<io.opentelemetry.sdk.resources.Entity> detectEntities()
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.detectors.ServiceInstanceDetector (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW INTERFACE: io.opentelemetry.sdk.resources.EntityDetector
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW FIELD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.resources.EntityDetector INSTANCE
+++ NEW METHOD: PUBLIC(+) java.util.List<io.opentelemetry.sdk.resources.Entity> detectEntities()
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.detectors.TelemetrySdkDetector (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW INTERFACE: io.opentelemetry.sdk.resources.EntityDetector
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW FIELD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.resources.detectors.TelemetrySdkDetector INSTANCE
+++ NEW METHOD: PUBLIC(+) java.util.List<io.opentelemetry.sdk.resources.Entity> detectEntities()
+++ NEW CLASS: PUBLIC(+) ABSTRACT(+) io.opentelemetry.sdk.resources.Entity (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW CONSTRUCTOR: PUBLIC(+) Entity()
+++ NEW METHOD: PUBLIC(+) STATIC(+) FINAL(+) io.opentelemetry.sdk.resources.EntityBuilder builder()
+++ NEW METHOD: PUBLIC(+) ABSTRACT(+) io.opentelemetry.api.common.Attributes getAttributes()
+++ NEW METHOD: PUBLIC(+) ABSTRACT(+) io.opentelemetry.api.common.Attributes getIdentifyingAttributes()
+++ NEW METHOD: PUBLIC(+) ABSTRACT(+) java.lang.String getSchemaUrl()
+++ NEW ANNOTATION: javax.annotation.Nullable
+++ NEW METHOD: PUBLIC(+) ABSTRACT(+) java.lang.String getType()
+++ NEW METHOD: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.EntityBuilder toBuilder()
+++ NEW CLASS: PUBLIC(+) io.opentelemetry.sdk.resources.EntityBuilder (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.Entity build()
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.EntityBuilder setEntityType(java.lang.String)
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.EntityBuilder setSchemaUrl(java.lang.String)
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.EntityBuilder withDescriptive(java.util.function.Consumer<io.opentelemetry.api.common.AttributesBuilder>)
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.EntityBuilder withIdentifying(java.util.function.Consumer<io.opentelemetry.api.common.AttributesBuilder>)
+++ NEW INTERFACE: PUBLIC(+) ABSTRACT(+) io.opentelemetry.sdk.resources.EntityDetector (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW METHOD: PUBLIC(+) ABSTRACT(+) java.util.List<io.opentelemetry.sdk.resources.Entity> detectEntities()
**** MODIFIED CLASS: PUBLIC ABSTRACT io.opentelemetry.sdk.resources.Resource (not serializable)
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
+++ NEW METHOD: PUBLIC(+) STATIC(+) io.opentelemetry.sdk.resources.Resource create(io.opentelemetry.api.common.Attributes, java.lang.String, java.util.Collection<io.opentelemetry.sdk.resources.Entity>)
*** MODIFIED METHOD: PUBLIC NON_ABSTRACT (<- ABSTRACT) io.opentelemetry.api.common.Attributes getAttributes()
+++* NEW METHOD: PUBLIC(+) ABSTRACT(+) java.util.Collection<io.opentelemetry.sdk.resources.Entity> getEntities()
*** MODIFIED CLASS: PUBLIC io.opentelemetry.sdk.resources.ResourceBuilder (not serializable)
=== CLASS FILE FORMAT VERSION: 52.0 <- 52.0
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.ResourceBuilder add(io.opentelemetry.sdk.resources.Entity)
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.ResourceBuilder addAll(java.util.Collection<io.opentelemetry.sdk.resources.Entity>)
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.ResourceProvider (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW METHOD: PUBLIC(+) STATIC(+) io.opentelemetry.sdk.resources.ResourceProviderBuilder builder()
+++ NEW METHOD: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.Resource getResource()
+++ NEW CLASS: PUBLIC(+) FINAL(+) io.opentelemetry.sdk.resources.ResourceProviderBuilder (not serializable)
+++ CLASS FILE FORMAT VERSION: 52.0 <- n.a.
+++ NEW SUPERCLASS: java.lang.Object
+++ NEW CONSTRUCTOR: PUBLIC(+) ResourceProviderBuilder()
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.ResourceProviderBuilder addDetectedResource(io.opentelemetry.sdk.resources.Resource)
+++ NEW ANNOTATION: java.lang.Deprecated
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.ResourceProviderBuilder addEntityDetector(io.opentelemetry.sdk.resources.EntityDetector)
+++ NEW METHOD: PUBLIC(+) io.opentelemetry.sdk.resources.ResourceProvider build()
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,22 @@ public void writeString(
generator.writeString(string);
}

@Override
public void writeRepeatedString(ProtoFieldInfo field, byte[][] utf8Bytes) throws IOException {
generator.writeArrayFieldStart(field.getJsonName());
for (byte[] value : utf8Bytes) {
// Marshalers encoded String into UTF-8 bytes to optimize for binary serialization where
// we are able to avoid the encoding process happening twice, one for size computation and one
// for actual writing. JsonGenerator actually has a writeUTF8String that would be able to
// accept
// this, but it only works when writing to an OutputStream, but not to a String like we do for
// writing to logs. It's wasteful to take a String, convert it to bytes, and convert back to
// the same String but we can see if this can be improved in the future.
generator.writeString(new String(value, StandardCharsets.UTF_8));
}
generator.writeEndArray();
}

@Override
public void writeBytes(ProtoFieldInfo field, byte[] value) throws IOException {
generator.writeBinaryField(field.getJsonName(), value);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,13 @@ public void writeString(
StatelessMarshalerUtil.writeUtf8(output, string, utf8Length, context);
}

@Override
public void writeRepeatedString(ProtoFieldInfo field, byte[][] utf8Bytes) throws IOException {
for (byte[] value : utf8Bytes) {
writeString(field, value);
}
}

@Override
public void writeBytes(ProtoFieldInfo field, byte[] value) throws IOException {
output.writeUInt32NoTag(field.getTag());
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,18 @@ public void serializeString(ProtoFieldInfo field, byte[] utf8Bytes) throws IOExc
writeString(field, utf8Bytes);
}

/**
* Serializes a protobuf {@code repeated string} field. {@code utf8Bytes} is the UTF8 encoded
* bytes of the strings to serialize.
*/
@SuppressWarnings("AvoidObjectArrays")
public void serializeRepeatedString(ProtoFieldInfo field, byte[][] utf8Bytes) throws IOException {
if (utf8Bytes.length == 0) {
return;
}
writeRepeatedString(field, utf8Bytes);
}

/**
* Serializes a protobuf {@code string} field. {@code string} is the value to be serialized and
* {@code utf8Length} is the length of the string after it is encoded in UTF8. This method reads
Expand All @@ -246,6 +258,11 @@ public abstract void writeString(
ProtoFieldInfo field, String string, int utf8Length, MarshalerContext context)
throws IOException;

/** Writes a protobuf {@code repeated string} field, even if it matches the default value. */
@SuppressWarnings("AvoidObjectArrays")
public abstract void writeRepeatedString(ProtoFieldInfo field, byte[][] utf8Bytes)
throws IOException;

/** Serializes a protobuf {@code bytes} field. */
public void serializeBytes(ProtoFieldInfo field, byte[] value) throws IOException {
if (value.length == 0) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.internal.otlp;

import io.opentelemetry.api.internal.StringUtils;
import io.opentelemetry.exporter.internal.marshal.MarshalerUtil;
import io.opentelemetry.exporter.internal.marshal.MarshalerWithSize;
import io.opentelemetry.exporter.internal.marshal.Serializer;
import io.opentelemetry.exporter.internal.otlp.experimental.ResourceEntityRefExperimental;
import io.opentelemetry.sdk.resources.Entity;
import java.io.IOException;
import java.nio.charset.StandardCharsets;
import javax.annotation.Nullable;

/**
* A Marshaler of {@link io.opentelemetry.sdk.resources.Entity}.
*
* <p>This class is internal and is hence not for public use. Its APIs are unstable and can change
* at any time.
*/
public final class ResourceEntityRefMarshaler extends MarshalerWithSize {
@Nullable private final byte[] schemaUrlUtf8;
private final byte[] typeUtf8;
private final byte[][] identityAttributeKeysUtf8;
private final byte[][] descriptiveAttributeKeysUtf8;

@Override
protected void writeTo(Serializer output) throws IOException {
if (schemaUrlUtf8 != null) {
output.writeString(ResourceEntityRefExperimental.SCHEMA_URL, schemaUrlUtf8);
}
output.writeString(ResourceEntityRefExperimental.TYPE, typeUtf8);
output.writeRepeatedString(
ResourceEntityRefExperimental.IDENTITY_ATTRIBUTES, identityAttributeKeysUtf8);
output.writeRepeatedString(
ResourceEntityRefExperimental.DESCRIPTION_ATTRIBUTES, descriptiveAttributeKeysUtf8);
}

public static ResourceEntityRefMarshaler createForEntity(Entity e) {
byte[] schemaUrlUtf8 = null;
if (!StringUtils.isNullOrEmpty(e.getSchemaUrl())) {
schemaUrlUtf8 = e.getSchemaUrl().getBytes(StandardCharsets.UTF_8);
}
return new ResourceEntityRefMarshaler(
schemaUrlUtf8,
e.getType().getBytes(StandardCharsets.UTF_8),
e.getIdentifyingAttributes().asMap().keySet().stream()
.map(key -> key.getKey().getBytes(StandardCharsets.UTF_8))
.toArray(byte[][]::new),
e.getAttributes().asMap().keySet().stream()
.map(key -> key.getKey().getBytes(StandardCharsets.UTF_8))
.toArray(byte[][]::new));
}

private ResourceEntityRefMarshaler(
@Nullable byte[] schemaUrlUtf8,
byte[] typeUtf8,
byte[][] identityAttributeKeysUtf8,
byte[][] descriptiveAttributeKeysUtf8) {
super(
calculateSize(
schemaUrlUtf8, typeUtf8, identityAttributeKeysUtf8, descriptiveAttributeKeysUtf8));
this.schemaUrlUtf8 = schemaUrlUtf8;
this.typeUtf8 = typeUtf8;
this.identityAttributeKeysUtf8 = identityAttributeKeysUtf8;
this.descriptiveAttributeKeysUtf8 = descriptiveAttributeKeysUtf8;
}

private static int calculateSize(
@Nullable byte[] schemaUrlUtf8,
byte[] typeUtf8,
byte[][] identityAttributeKeysUtf8,
byte[][] descriptiveAttributeKeysUtf8) {
int size = 0;
if (schemaUrlUtf8 != null) {
size += MarshalerUtil.sizeBytes(ResourceEntityRefExperimental.SCHEMA_URL, schemaUrlUtf8);
}
size += MarshalerUtil.sizeBytes(ResourceEntityRefExperimental.TYPE, typeUtf8);
// TODO - we need repeated string support.
for (byte[] keyUtf8 : identityAttributeKeysUtf8) {
size += MarshalerUtil.sizeBytes(ResourceEntityRefExperimental.IDENTITY_ATTRIBUTES, keyUtf8);
}
for (byte[] keyUtf8 : descriptiveAttributeKeysUtf8) {
size +=
MarshalerUtil.sizeBytes(ResourceEntityRefExperimental.DESCRIPTION_ATTRIBUTES, keyUtf8);
}
return size;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import io.opentelemetry.exporter.internal.marshal.MarshalerUtil;
import io.opentelemetry.exporter.internal.marshal.MarshalerWithSize;
import io.opentelemetry.exporter.internal.marshal.Serializer;
import io.opentelemetry.exporter.internal.otlp.experimental.ResourceExperimental;
import io.opentelemetry.proto.resource.v1.internal.Resource;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
Expand Down Expand Up @@ -37,7 +38,10 @@ public static ResourceMarshaler create(io.opentelemetry.sdk.resources.Resource r

RealResourceMarshaler realMarshaler =
new RealResourceMarshaler(
KeyValueMarshaler.createForAttributes(resource.getAttributes()));
KeyValueMarshaler.createForAttributes(resource.getAttributes()),
resource.getEntities().stream()
.map(ResourceEntityRefMarshaler::createForEntity)
.toArray(MarshalerWithSize[]::new));

ByteArrayOutputStream binaryBos =
new ByteArrayOutputStream(realMarshaler.getBinarySerializedSize());
Expand Down Expand Up @@ -70,19 +74,30 @@ public void writeTo(Serializer output) throws IOException {

private static final class RealResourceMarshaler extends MarshalerWithSize {
private final KeyValueMarshaler[] attributes;
private final MarshalerWithSize[] entityRefs;

private RealResourceMarshaler(KeyValueMarshaler[] attributes) {
super(calculateSize(attributes));
private RealResourceMarshaler(KeyValueMarshaler[] attributes, MarshalerWithSize[] entityRefs) {
super(calculateSize(attributes, entityRefs));
this.attributes = attributes;
this.entityRefs = entityRefs;
}

@Override
protected void writeTo(Serializer output) throws IOException {
output.serializeRepeatedMessage(Resource.ATTRIBUTES, attributes);
if (entityRefs.length > 0) {
output.serializeRepeatedMessage(ResourceExperimental.ENTITY_REFS, entityRefs);
}
}

private static int calculateSize(KeyValueMarshaler[] attributeMarshalers) {
return MarshalerUtil.sizeRepeatedMessage(Resource.ATTRIBUTES, attributeMarshalers);
private static int calculateSize(
KeyValueMarshaler[] attributeMarshalers, MarshalerWithSize[] entityRefs) {
int size = 0;
size += MarshalerUtil.sizeRepeatedMessage(Resource.ATTRIBUTES, attributeMarshalers);
if (entityRefs.length > 0) {
size += MarshalerUtil.sizeRepeatedMessage(ResourceExperimental.ENTITY_REFS, entityRefs);
}
return size;
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.internal.otlp.experimental;

import io.opentelemetry.exporter.internal.marshal.ProtoFieldInfo;

/**
* This class is internal and is hence not for public use. Its APIs are unstable and can change at
* any time.
*/
public final class ResourceEntityRefExperimental {
public static final ProtoFieldInfo SCHEMA_URL = ProtoFieldInfo.create(1, 10, "schemaUrl");
public static final ProtoFieldInfo TYPE = ProtoFieldInfo.create(2, 18, "type");
public static final ProtoFieldInfo IDENTITY_ATTRIBUTES =
ProtoFieldInfo.create(3, 26, "idAttrKeys");
public static final ProtoFieldInfo DESCRIPTION_ATTRIBUTES =
ProtoFieldInfo.create(4, 34, "descrAttrKeys");

private ResourceEntityRefExperimental() {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.internal.otlp.experimental;

import io.opentelemetry.exporter.internal.marshal.ProtoFieldInfo;

/**
* This class is internal and is hence not for public use. Its APIs are unstable and can change at
* any time.
*/
public final class ResourceExperimental {
public static final ProtoFieldInfo ENTITY_REFS = ProtoFieldInfo.create(3, 26, "entityRefs");

private ResourceExperimental() {}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
/*
* Copyright The OpenTelemetry Authors
* SPDX-License-Identifier: Apache-2.0
*/

package io.opentelemetry.exporter.internal.otlp;

import static org.assertj.core.api.Assertions.assertThat;

import io.opentelemetry.sdk.resources.Entity;
import io.opentelemetry.sdk.resources.Resource;
import java.io.ByteArrayOutputStream;
import java.nio.charset.StandardCharsets;
import org.junit.jupiter.api.Test;

public class ResourceEntityTest {
@Test
void toJsonResourceWithEntity() throws Exception {
Resource resource =
Resource.builder()
.add(
Entity.builder()
.setSchemaUrl("http://example.com/1.0")
.setEntityType("test")
.withIdentifying(attr -> attr.put("test.id", 1))
.withDescriptive(attr -> attr.put("test.name", "one"))
.build())
.build();

ByteArrayOutputStream out = new ByteArrayOutputStream();
try {
ResourceMarshaler.create(resource).writeJsonTo(out);
} finally {
out.close();
}

String json = new String(out.toByteArray(), StandardCharsets.UTF_8);
assertThat(json)
.isEqualTo(
"{\"attributes\":[{\"key\":\"test.id\",\"value\":{\"intValue\":\"1\"}},{\"key\":\"test.name\",\"value\":{\"stringValue\":\"one\"}}],"
+ "\"entityRefs\":[{\"schemaUrl\":\"http://example.com/1.0\",\"type\":\"test\",\"idAttrKeys\":[\"test.id\"],\"descrAttrKeys\":[\"test.name\"]}]}");
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -412,21 +412,21 @@ void stringRepresentation() {
+ "tracerProvider=SdkTracerProvider{"
+ "clock=SystemClock{}, "
+ "idGenerator=RandomIdGenerator{}, "
+ "resource=Resource{schemaUrl=null, attributes={service.name=\"otel-test\"}}, "
+ "resource=Resource{schemaUrl=null, rawAttributes={service.name=\"otel-test\"}, entities=[]}, "
+ "spanLimitsSupplier=SpanLimitsValue{maxNumberOfAttributes=128, maxNumberOfEvents=128, maxNumberOfLinks=128, maxNumberOfAttributesPerEvent=128, maxNumberOfAttributesPerLink=128, maxAttributeValueLength=2147483647}, "
+ "sampler=ParentBased{root:AlwaysOnSampler,remoteParentSampled:AlwaysOnSampler,remoteParentNotSampled:AlwaysOffSampler,localParentSampled:AlwaysOnSampler,localParentNotSampled:AlwaysOffSampler}, "
+ "spanProcessor=SimpleSpanProcessor{spanExporter=MultiSpanExporter{spanExporters=[MockSpanExporter{}, MockSpanExporter{}]}, exportUnsampledSpans=false}"
+ "}, "
+ "meterProvider=SdkMeterProvider{"
+ "clock=SystemClock{}, "
+ "resource=Resource{schemaUrl=null, attributes={service.name=\"otel-test\"}}, "
+ "resource=Resource{schemaUrl=null, rawAttributes={service.name=\"otel-test\"}, entities=[]}, "
+ "metricReaders=[PeriodicMetricReader{exporter=MockMetricExporter{}, intervalNanos=60000000000}], "
+ "metricProducers=[], "
+ "views=[RegisteredView{instrumentSelector=InstrumentSelector{instrumentName=instrument}, view=View{name=new-instrument, aggregation=DefaultAggregation, attributesProcessor=NoopAttributesProcessor{}, cardinalityLimit=2000}}]"
+ "}, "
+ "loggerProvider=SdkLoggerProvider{"
+ "clock=SystemClock{}, "
+ "resource=Resource{schemaUrl=null, attributes={service.name=\"otel-test\"}}, "
+ "resource=Resource{schemaUrl=null, rawAttributes={service.name=\"otel-test\"}, entities=[]}, "
+ "logLimits=LogLimits{maxNumberOfAttributes=128, maxAttributeValueLength=2147483647}, "
+ "logRecordProcessor=SimpleLogRecordProcessor{logRecordExporter=MultiLogRecordExporter{logRecordExporters=[MockLogRecordExporter{}, MockLogRecordExporter{}]}}"
+ "}, "
Expand Down
Loading