Appendices

Table of Contents

Appendix A: Frequently Asked Questions
Appendix B: Observability
- B.1. Observability metadata

Appendix A: Frequently Asked Questions

A.1. Connection to the proxy cannot be established

Netty’s HTTP proxy support always uses CONNECT method in order to establish a tunnel to the specified proxy regardless of the scheme that is used http or https. (More information: Netty enforce HTTP proxy to support HTTP CONNECT method). Some proxies might not support CONNECT method when the scheme is http or might need to be configured in order to support this way of communication. Sometimes this might be the reason for not being able to connect to the proxy. Consider checking the proxy documentation whether it supports or needs an additional configuration in order to support CONNECT method.

A.2. What is the meaning of the information that is prepended to every log record?

Reactor Netty adds information for the connection at the beginning of every log record (when this is possible). There is a slight difference in the details for the connection when you use TCP, UDP, HTTP/1.1 or HTTP/2.

A.2.1. TCP and UDP

In case of TCP and UDP, the following is added at the beginning of every log record: the id of the underlying connection, local and remote addresses.

Examples
[a1566d55, L:/[0:0:0:0:0:0:0:1]:53446 - R:/[0:0:0:0:0:0:0:1]:53444]
[a1566d55, L:/[0:0:0:0:0:0:0:1]:53446 ! R:/[0:0:0:0:0:0:0:1]:53444]

Format
[<CONNECTION_ID>, L:<LOCAL_ADDRESS> <CONNECTION_OPENED_CLOSED> R:<REMOTE_ADDRESS>]
<CONNECTION_ID>: a1566d55
<LOCAL_ADDRESS>: [0:0:0:0:0:0:0:1]:53446
<CONNECTION_OPENED_CLOSED>: - (connection opened)
                            ! (connection closed)
<REMOTE_ADDRESS>: [0:0:0:0:0:0:0:1]:53444

A.2.2. HTTP/1.1

In case of HTTP/1.1, the following is added at the beginning of every log record: the id of the underlying connection, the serial number of the request received on that connection, local and remote addresses.

Examples
[a1566d55-5, L:/[0:0:0:0:0:0:0:1]:53446 - R:/[0:0:0:0:0:0:0:1]:53444]
[a1566d55-5, L:/[0:0:0:0:0:0:0:1]:53446 ! R:/[0:0:0:0:0:0:0:1]:53444]

Format
[<CONNECTION_ID>-<REQUEST_NUMBER>, L:<LOCAL_ADDRESS> <CONNECTION_OPENED_CLOSED> R:<REMOTE_ADDRESS>]
<CONNECTION_ID>: a1566d55
<REQUEST_NUMBER>: 5
<LOCAL_ADDRESS>: [0:0:0:0:0:0:0:1]:53446
<CONNECTION_OPENED_CLOSED>: - (connection opened)
                            ! (connection closed)
<REMOTE_ADDRESS>: [0:0:0:0:0:0:0:1]:53444

A.2.3. HTTP/2

In case of HTTP/2, the following is added at the beginning of every log record: the id of the underlying connection, local and remote addresses, the id of the stream received on that connection.

Examples
[a1566d55, L:/[0:0:0:0:0:0:0:1]:53446 - R:/[0:0:0:0:0:0:0:1]:53444](H2 - 5)
[a1566d55, L:/[0:0:0:0:0:0:0:1]:53446 ! R:/[0:0:0:0:0:0:0:1]:53444](H2 - 5)

Format
[<CONNECTION_ID>, L:<LOCAL_ADDRESS> <CONNECTION_OPENED_CLOSED> R:<REMOTE_ADDRESS>]<STREAM_ID>
<CONNECTION_ID>: a1566d55
<LOCAL_ADDRESS>: [0:0:0:0:0:0:0:1]:53446
<CONNECTION_OPENED_CLOSED>: - (connection opened)
                            ! (connection closed)
<REMOTE_ADDRESS>: [0:0:0:0:0:0:0:1]:53444
<STREAM_ID>: (H2 - 5)

A.3. How can I extract all log records for a particular HTTP request?

Reactor Netty adds information for the connection at the beginning of every log record (when this is possible). Use the id of the connection in order to extract all log records for a particular HTTP request. For more information see What is the meaning of the information that is prepended to every log record?

A.4. How can I debug a memory leak?

By default, Reactor Netty uses direct memory as this is more performant when there are many native I/O operations (working with sockets), as this can remove the copying operations. As allocation and deallocation are expensive operations, Reactor Netty also uses pooled buffers by default. For more information, see Reference Counted Objects.

To be able to debug memory issues with the direct memory and the pooled buffers, Netty provides a special memory leak detection mechanism. Follow the instructions for Troubleshooting Buffer Leaks to enable this mechanism. In addition to the instructions provided by Netty, Reactor Netty provides a special logger (_reactor.netty.channel.LeakDetection) that helps to identify where the memory leak might be located inside Reactor Netty or whether Reactor Netty already forwarded the ownership of the buffers to the application/framework. By default, this logger is disabled. To enable it, increase the log level to DEBUG.

Another way to detect memory leaks is to monitor reactor.netty.bytebuf.allocator.active.heap.memory and reactor.netty.bytebuf.allocator.active.direct.memory meters:

The reactor.netty.bytebuf.allocator.active.heap.memory provides the actual bytes consumed by in-use buffers allocated from heap buffer pools
The reactor.netty.bytebuf.allocator.active.direct.memory provides the actual bytes consumed by in-use buffers allocated from direct buffer pools

If the above meters are constantly growing, then it’s likely that there is a buffer memory leak.

Consider reducing the used memory when debugging memory leak issues (e.g -XX:MaxDirectMemorySize, -Xms, -Xmx). The less memory the application has, the sooner the memory leak will happen.

A.5. How can I debug "Connection prematurely closed BEFORE response"?

By default, Reactor Netty clients use connection pooling. When a connection is acquired from the connection pool, it is checked to see whether it is still open. However, the connection can be closed at any time after the acquisition. There are many reasons that can cause a connection to be closed. In most cases, the client might not send directly to the server. Instead, there might be other network components (proxies, load balancers, and so on) between them.

If, on the client side, you observe Connection prematurely closed BEFORE response, perform the following checks to identify the reason for the connection being closed:

Obtain a TCP dump and check which peer sends a FIN/RST signal.
Check your network connection.
Check your Firewall and VPN.
Check for any proxies and load balancers.
- Do they have some kind of idle timeout configuration (the connection is closed when there is no incoming data for a certain period of time)?
- Do they silently drop the idle connections without sending any signal? In order to verify whether this might be the issue, you can enable the TCP keep-alive as described in the section Connection Timeout. Issues related to TCP keep-alive configuration on various load balancers were reported in the past.
Check the target server.
- Are there configurations related to any of the following?
  - idle timeout (the connection is closed when there is no incoming data for a certain period of time)
  - limit for buffering data in memory
  - multipart exceeds the max file size limit
  - bad request
  - max keep alive requests (the connection is closed when the requests reach the configured maximum number)
  - rate limit configuration
- Is the target server in a shutting down state?

Consider checking Timeout Configuration. The section describes various timeout configuration options that are available for Reactor Netty clients. Configuring a proper timeout may improve or solve issues in the communication process.

Appendix B: Observability

B.1. Observability metadata

B.1.1. Observability - Metrics

Below you can find a list of all metrics declared by this project.

Active Connections

The number of the connections in the connection pool that have been successfully acquired and are in active use.

Metric name reactor.netty.connection.provider.active.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.client.Http2ConnectionProviderMeters.

Table 1. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Active Connections

The number of the connections in the connection pool that have been successfully acquired and are in active use.

Metric name reactor.netty.connection.provider.active.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 2. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Active Direct Memory

The actual bytes consumed by in-use buffers allocated from direct buffer pools.

Metric name reactor.netty.bytebuf.allocator.active.direct.memory. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 3. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Active Heap Memory

The actual bytes consumed by in-use buffers allocated from heap buffer pools.

Metric name reactor.netty.bytebuf.allocator.active.heap.memory. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 4. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Active Streams

The number of the active HTTP/2 streams.

Metric name reactor.netty.connection.provider.active.streams. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.client.Http2ConnectionProviderMeters.

Table 5. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Chunk Size

The chunk size for an arena.

Metric name reactor.netty.bytebuf.allocator.chunk.size. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 6. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Connections Active

The number of http connections, on the server, currently processing requests.

Metric name reactor.netty.http.server.connections.active. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 7. Low cardinality Keys
Name	Description
`local.address` (required)	Local address.
`uri` (required)	URI.

Connections Total

The number of all opened connections on the server.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.channel.ChannelMeters.

Table 8. Low cardinality Keys
Name	Description
`local.address` (required)	Local address.
`uri` (required)	URI.

Connect Time

Connect metric.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type timer.

Metric name %s.active - since it contains %s, the name is dynamic and will be resolved at runtime. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.channel.ConnectObservations.

Table 9. Low cardinality Keys
Name	Description
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.

Table 10. High cardinality Keys
Name	Description
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Data Received

Amount of the data received, in bytes.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type distribution summary and base unit bytes.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.channel.ChannelMeters.

Table 11. Low cardinality Keys
Name	Description
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`uri` (required)	URI.

Data Sent

Amount of the data sent, in bytes.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type distribution summary and base unit bytes.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.channel.ChannelMeters.

Table 12. Low cardinality Keys
Name	Description
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`uri` (required)	URI.

Direct Arenas

The number of direct arenas.

Metric name reactor.netty.bytebuf.allocator.direct.arenas. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 13. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Errors Count

Number of errors that occurred.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type counter.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.channel.ChannelMeters.

Table 14. Low cardinality Keys
Name	Description
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`uri` (required)	URI.

Heap Arenas

The number of heap arenas.

Metric name reactor.netty.bytebuf.allocator.heap.arenas. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 15. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Hostname Resolution Time

Hostname resolution metric.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type timer.

Metric name %s.active - since it contains %s, the name is dynamic and will be resolved at runtime. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.transport.HostnameResolutionObservations.

Table 16. Low cardinality Keys
Name	Description
`remote.address` (required)	Remote address.
`status` (required)	STATUS.

Table 17. High cardinality Keys
Name	Description
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Http Client Data Received Time

Time spent in consuming incoming data on the client.

Metric name reactor.netty.http.client.data.received.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.client.HttpClientMeters.

Table 18. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.
`uri` (required)	URI.

Http Client Data Sent Time

Time spent in sending outgoing data from the client.

Metric name reactor.netty.http.client.data.sent.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.client.HttpClientMeters.

Table 19. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`uri` (required)	URI.

Http Client Response Time

Response metric.

Metric name reactor.netty.http.client.response.time. Type timer.

Metric name reactor.netty.http.client.response.time.active. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.client.HttpClientObservations.

Table 20. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.
`uri` (required)	URI.

Table 21. High cardinality Keys
Name	Description
`http.status_code` (required)	Status code.
`http.url` (required)	URL.
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Http Server Data Received

Amount of the data received, in bytes.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type distribution summary and base unit bytes.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 22. Low cardinality Keys
Name	Description
`uri` (required)	URI.

Http Server Data Received Time

Time spent in consuming incoming data on the server.

Metric name reactor.netty.http.server.data.received.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 23. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`uri` (required)	URI.

Http Server Data Sent

Amount of the data sent, in bytes.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type distribution summary and base unit bytes.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 24. Low cardinality Keys
Name	Description
`uri` (required)	URI.

Http Server Data Sent Time

Time spent in sending outgoing data from the server.

Metric name reactor.netty.http.server.data.sent.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 25. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`status` (required)	STATUS.
`uri` (required)	URI.

Http Server Errors Count

Number of errors that occurred.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type counter.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 26. Low cardinality Keys
Name	Description
`uri` (required)	URI.

Http Server Response Time

Response metric.

Metric name reactor.netty.http.server.response.time. Type timer.

Metric name reactor.netty.http.server.response.time.active. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerObservations.

Table 27. Low cardinality Keys
Name	Description
`method` (required)	METHOD.
`status` (required)	STATUS.
`uri` (required)	URI.

Table 28. High cardinality Keys
Name	Description
`http.scheme` (required)	HTTP scheme.
`http.status_code` (required)	Status code.
`net.host.name` (required)	Net host name.
`net.host.port` (required)	Net host port.
`reactor.netty.type` (required)	Reactor Netty type (always server).

Idle Connections

The number of the idle connections in the connection pool.

Metric name reactor.netty.connection.provider.idle.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.client.Http2ConnectionProviderMeters.

Table 29. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Idle Connections

The number of the idle connections in the connection pool.

Metric name reactor.netty.connection.provider.idle.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 30. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Max Connections

The maximum number of active connections that are allowed in the connection pool.

Metric name reactor.netty.connection.provider.max.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 31. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Max Pending Connections

The maximum number of requests that will be queued while waiting for a ready connection from the connection pool.

Metric name reactor.netty.connection.provider.max.pending.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 32. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Normal Cache Size

The size of the normal cache.

Metric name reactor.netty.bytebuf.allocator.normal.cache.size. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 33. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Pending Connections

The number of the request, that are pending acquire a connection from the connection pool.

Metric name reactor.netty.connection.provider.pending.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 34. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Pending Connections Time

Time spent in pending acquire a connection from the connection pool.

Metric name reactor.netty.connection.provider.pending.connections.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 35. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.

Pending Streams

The number of requests that are waiting for opening HTTP/2 stream.

Metric name reactor.netty.connection.provider.pending.streams. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.client.Http2ConnectionProviderMeters.

Table 36. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Pending Streams Time

Time spent in pending acquire a stream from the connection pool.

Metric name reactor.netty.connection.provider.pending.streams.time. Type timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.http.client.Http2ConnectionProviderMeters.

Table 37. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.

Pending Tasks

Event loop pending scheduled tasks.

Metric name reactor.netty.eventloop.pending.tasks. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.EventLoopMeters.

Table 38. Low cardinality Keys
Name	Description
`name` (required)	NAME.

Small Cache Size

The size of the small cache.

Metric name reactor.netty.bytebuf.allocator.small.cache.size. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 39. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Streams Active

The number of HTTP/2 streams currently active on the server.

Metric name reactor.netty.http.server.streams.active. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerMeters.

Table 40. Low cardinality Keys
Name	Description
`local.address` (required)	Local address.
`uri` (required)	URI.

Thread Local Caches

The number of thread local caches.

Metric name reactor.netty.bytebuf.allocator.threadlocal.caches. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 41. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Tls Handshake Time

TLS handshake metric.

Metric name %s - since it contains %s, the name is dynamic and will be resolved at runtime. Type timer.

Metric name %s.active - since it contains %s, the name is dynamic and will be resolved at runtime. Type long task timer.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Micrometer internally uses nanoseconds for the baseunit. However, each backend determines the actual baseunit. (i.e. Prometheus uses seconds)

Fully qualified name of the enclosing class reactor.netty.tcp.TlsHandshakeObservations.

Table 42. Low cardinality Keys
Name	Description
`proxy.address` (required)	Proxy address, when there is a proxy configured.
`remote.address` (required)	Remote address.
`status` (required)	STATUS.

Table 43. High cardinality Keys
Name	Description
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (client/server).

Total Connections

The number of all connections in the connection pool, active or idle.

Metric name reactor.netty.connection.provider.total.connections. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.resources.ConnectionProviderMeters.

Table 44. Low cardinality Keys
Name	Description
`id` (required)	ID.
`name` (required)	NAME.
`remote.address` (required)	Remote address.

Used Direct Memory

The number of bytes reserved by direct buffer allocator.

Metric name reactor.netty.bytebuf.allocator.used.direct.memory. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 45. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

Used Heap Memory

The number of bytes reserved by heap buffer allocator.

Metric name reactor.netty.bytebuf.allocator.used.heap.memory. Type gauge.

KeyValues that are added after starting the Observation might be missing from the *.active metrics.

Fully qualified name of the enclosing class reactor.netty.transport.ByteBufAllocatorMeters.

Table 46. Low cardinality Keys
Name	Description
`id` (required)	ID.
`type` (required)	TYPE.

B.1.2. Observability - Spans

Below you can find a list of all spans declared by this project.

Connect Span

Connect Span.

Span name %s - since it contains %s, the name is dynamic and will be resolved at runtime.

Fully qualified name of the enclosing class reactor.netty.channel.ConnectSpans.

Table 47. Tag Keys
Name	Description
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Hostname Resolution Span

Hostname Resolution Span.

Span name %s - since it contains %s, the name is dynamic and will be resolved at runtime.

Fully qualified name of the enclosing class reactor.netty.transport.HostnameResolutionSpans.

Table 48. Tag Keys
Name	Description
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Http Client Response Span

Response Span.

Span name %s - since it contains %s, the name is dynamic and will be resolved at runtime.

Fully qualified name of the enclosing class reactor.netty.http.client.HttpClientSpans.

Table 49. Tag Keys
Name	Description
`http.status_code` (required)	Status code.
`http.url` (required)	URL.
`net.peer.name` (required)	Net peer name.
`net.peer.port` (required)	Net peer port.
`reactor.netty.type` (required)	Reactor Netty type (always client).

Http Server Response Span

Response Span.

Span name %s - since it contains %s, the name is dynamic and will be resolved at runtime.

Fully qualified name of the enclosing class reactor.netty.http.server.HttpServerSpans.

Table 50. Tag Keys
Name	Description
`http.scheme` (required)	HTTP scheme.
`http.status_code` (required)	Status code.
`net.host.name` (required)	Net host name.
`net.host.port` (required)	Net host port.
`reactor.netty.type` (required)	Reactor Netty type (always server).

Tls Handshake Span

TLS Handshake Span.

Span name %s - since it contains %s, the name is dynamic and will be resolved at runtime.

Fully qualified name of the enclosing class reactor.netty.tcp.TlsHandshakeSpans.

Table 51. Tag Keys
Name	Description
`reactor.netty.protocol` (required)	Reactor Netty protocol (tcp/http etc.).
`reactor.netty.status` (required)	Reactor Netty status.
`reactor.netty.type` (required)	Reactor Netty type (client/server).
`remote.address` (required)	Remote address.