Frustrated by Magento? Then you’ll love
- What is a Zipkin Trace?
- Instrumenting Traces with Zipkin
- Tracing NodeJS Services with Open Telemetry
- OpenTelemetry’s NPM Packages
In our previous OpenTelemetry articles, we gave a broad “hello world” overview of how an end-user-programmer might use OpenTelemetry to instrument their programs and generate metrics and traces.
Today we’re going to take a look at the entire OpenTelemetry Javascript project, as it exists in mid-2020. This article should help you understand the project as a whole better, and where certain functionality lives. Some of the categorization we’ll be doing might not line up 100% with the official project specs. We’ll try to make note of this when we’re doing it.
Also, at the risk of breaking the omnipresent all-seeing narrator bit, — I’m relatively new to the OpenTelemetry project myself and may have missed something. If you spot an error don’t be shy about saying hello and letting me know.
OpenTelemetry Packages: Core and Contrib
The OpenTelemetry project distributes its javascript code via NPM Packages under the @opentelemetry namespace.
This code comes from two distinct GitHub “mono-repositories”. A mono-repository is a single GitHub repository that contains many different packages. OpenTelemetry uses Lerna to manage their mono-repositories. Lerna is a tool that helps you manage and publish multiple NPM packages from one repository.
You can find these two repositories at
The first, opentelemetry-js, is the core repository. The packages here are the packages that are required for OpenTelemetry to do its thing. No single team will use all these packages, but they do form the foundation of the system.
The second, opentelemetry-js-contrib, is the “contributors” repository. The OpenTelemetry project describes the packages in this repository as
[packages] that are not part of the core repository and core distribution of the API and SDK
— which is almost a Tautology. My current understanding of the contributors repository is that it’s for packages that provide optional functionality that targets specific frameworks. For example, if you you look at the Node plugins that are available it’s mostly instrumentation that adds spans measuring the performance of specific frameworks. It also seems like the contrib repository is where folks works on things when they’re not sure whether they’ll be included as part of the core project or not.
Repository Layout
If you’re used to working in a “one repository == one package” world, mono-repositories can take a little getting used to.
For Lerna managed mono-repositories, the first place you’ll want to stop is the pacakges key in the root level lerna.js file. This configuration section defines which folders in the repository are considered stand-alone packages.
If we look at the current (summer of 2020) lerna.js for the core repository we’ll see this
"packages": [
"benchmark/*",
"metapackages/*",
"packages/*"
],
This tells us (and, more importantly, tells the Lerna command line tool) that every top level folder in benchmark, metapackages, and packages is a single package.
Similarly, for the contrib repository, we have this
"packages": [
"packages/*",
"plugins/node/*",
"plugins/web/*",
"propagators/*"
],
Here we have another generic packages folder, but auto-instrumentation plugins and propagators get their own folders, and the auto-instrumentation plugins are further organized into plugins intended for NodeJS, and plugins intended for browsers.
Package Types
Packages can be broadly grouped into eight different categories. Some of these categories are using the official OpenTelemetry designations, and others are implicit categories that can make the project more understandable. Also, these categories don’t include the metapackages and benchmark folders mentioned above. A metapackage is a package that includes no code and is only there to pull in a specific set of other packages. The benchmark package has tools for testing the performance of of various other packages.
The eight categories we’ll look at today are: Automatic Instrumentations, Plugins, Software Development Kits, Foundational Packages, Exporters, Asynchronous Context Managers, Propagators, and Compatibility Packages
Automatic Instrumentation
At the top of everything are the auto-instrumentation packages. These are packages that end-user-programmers will add to their code when they want their programs traced.
The @opentelemetry/node package provides a way to trace NodeJS programs. The @opentelemetry/web package provides a way to trace browser based javascript programs.
Plugins
Then there’s the auto-instrumentation plugin packages, or in official OpenTelemetry terms, plugins. The automatic-instrumentation packages load these plugins automatically.
Auto-instrumentation plugins provide code that links up service requests with trace IDs and generates spans. In the future they may also generate metrics, but what role metrics play in OpenTelemetry is still being sorted out.
The core auto-instrumentation plugin packages are mostly about ensuring a trace ID is maintained between each request and response when a program uses various javascript modules to communicate with another program via HTTP or GRPC
- @opentelemetry/plugin-grpc
- @opentelemetry/plugin-http
- @opentelemetry/plugin-https
- @opentelemetry/plugin-xml-http-request
The contrib auto-instrumentation plugin packages are mostly about measuring the performance of popular javascript modules by generating spans.
- @opentelemetry/plugin-dns
- @opentelemetry/plugin-express
- @opentelemetry/plugin-ioredis
- @opentelemetry/plugin-mongodb
- @opentelemetry/plugin-mysql
- @opentelemetry/plugin-pg-pool
- @opentelemetry/plugin-pg
- @opentelemetry/plugin-redis
- @opentelemetry/plugin-document-load
- @opentelemetry/plugin-user-interaction
If you’re curious how to use these packages, Part three of this series shows how to get started with the NodeJS automatic instrumentation and plugins.
Software Development Kits
Moving one step down the stack there’s the the software development kits, or SDKs. “SDK” is an official bit of OpenTelemetry nomenclature — these are the packages that provide code that allows programmers to create the span and metric objects and measure things.
The @opentelemetry/metrics package contains code for generating metrics. The @opentelemetry/tracing package contains code for generating spans.
It’s worth mentioning that the OpenTelemetry specifications have specific ideas on what they consider an API or an SDK, and that (in June of 2020) the metrics portion of OpenTelemetry currently has no specced “SDK” for metrics. It’s not clear if metrics code will eventually be split out into a metrics “API” and “SDK” — but I’ve found it useful to think of both @opentelemetry/metrics and @opentelemetry/tracing as SDKs. These are the packages a plugin author uses to create telemetry data.
Foundational Packages
At the bottom of the stack there’s a few foundational packages — these are the packages that everything else sits on top of. They provide base classes, functions, data, and other functionality that all the other OpenTelemetry packages rely on.
It’s worth noting, again, that the OpenTelemetry project has a specification on what a “tracing API” is, and a specification that outlines what a resource SDK is. They don’t have any concept of a “foundational package” — the grouping of these three packages together is more about the reality of how the rest of the project uses this code than it is any specific behavior.
Exporter Packages
Exporter packages send telemetry data produced with the SDKs along to other services. At the time of this writing there are four exporters provided by the core project, and none provided by the contrib project.
- @opentelemetry/exporter-zipkin
- @opentelemetry/exporter-jaeger
- @opentelemetry/exporter-prometheus
- @opentelemetry/exporter-collector
The @opentelemetry/exporter-zipkin and @opentelemetry/exporter-jaeger exporters send spans to external Zipkin or Jaeger systems. The @opentelemetry/exporter-prometheus exporter sends metric data to systems running Prometheus. The @opentelemetry/exporter-collector exporter sends data to systems running the OpenTelemetry collector service.
It’s also worth noting that an exporter doesn’t need to be in this repository in order to work. The contrib exporters are the exporters for open source systems that the OpenTelemetry project decides are important enough to be there. There’s nothing stopping you (or commercial vendors) from writing an exporter for your own observability system and hosting it in your own repository.
Asynchronous Context Managers
OpenTelemetry tracing code uses asynchronous context manager packages to keep track of the current trace across NodeJS’s infamous asynchronous boundaries. i.e. These packages help programmers answer the question: When a program is making and processing multiple network requests at what seems like the same time, how do we known what the current trace actually is?
- @opentelemetry/context-base
- @opentelemetry/context-async-hooks
- @opentelemetry/context-zone-peer-dep
- @opentelemetry/context-zone
The @opentelemetry/context-base packages provides an interface for context classes to implement. The @opentelemetry/context-async-hooks package is the context manager for NodeJS programs. The @opentelemetry/context-zone and @opentelemetry/context-zone-per-dep packages provide a context manager suitable for use in a web browser.
Propagators
The OpenTelemetry project uses propagator packages to read and write information (like trace IDs) to/from network request headers so they can be shared between two systems communicating with each other via the network. Propagators are a little tricky in that stand-alone packages exist only in the contrib repository, but there are propagators in the core project
The @opentelemetry/propagator-jaeger package contains logic for reading and writing trace header information in Jaeger format. The @opentelemetry/propagator-grpc-census-binary packages contains logic for reading and writing headers in the OpenCensus grpc-trace-bin binary header format.
The core OpenTelemetry project doesn’t have propagator packages, but it does have propagators. The project just distributes them as part of the@opentelemetry/core package. The B3 Propagator contains logic for reading and writing headers in Zipkin’s popular B3 format. The HttpTraceContext propagator contains logic for reading and writing headers in the newer W3C Trace Context format.
Compatibility
Finally, there’s compatibility packages — or more appropriately, package.
Before there was an OpenTelemetry project, there was a project called OpenTracing and a project calls OpenCensus. The first concerned itself mostly with spans, and the second mostly with metrics. When the projects combined into OpenTelemetry there were promises made about backwards compatibility layers — @opentelemetry/shim-opentracing appears to be a shim layer between the old Open Tracing project and OpenTelemetry.
Wrap Up
That about covers it — if you got a little lost in there, don’t worry, It can be a lot to take in. In future series we intend to explore the use and implementation of many of these packages, as well as explain how they interact with each other.