# Sending Traces and Metrics to the OpenTelemetry Collector

In this example, we will show you an example configuration
for enabling the lifecycle-operator to send OpenTelemetry traces and metrics to the
[OpenTelemetry Collector](https://github.com/open-telemetry/opentelemetry-collector).
The Collector will then be used to forward the gathered data to
[Jaeger](https://www.jaegertracing.io)
and [Prometheus](https://prometheus.io).
The application deployed uses an example of pre-Deployment Evaluation
based on prometheus metrics.

- To install the whole demo, including Keptn,
  execute the following command:

  ```shell
  make install
  ```

- Deploy the PodTatoHead Demo Application: `make deploy-podtatohead`
- Afterward, see it in action as defined in
  [OpenTelemetry in Action](#seeing-the-opentelemetry-collector-in-action)

For information about installing and configuring
the software required, see
[OpenTelemetry observability](https://keptn.sh/latest/docs/guides/otel/)
in the documentation.

## Seeing the OpenTelemetry Collector in action

After everything has been set up, use the lifecycle operator
to deploy a workload (e.g. using the `single-service`
or `podtato-head` example in the `examples` folder).
To showcase pre-Evaluation checks,
we created a new version of podtato-head app in
assets/podtetohead-deployment-evaluation.
You can run ``make deploy-podtatohead``
to check pre-Evaluations of prometheus metrics
both at app and workload instance level.
Once an example has been deployed,
you can view the generated traces in Jaeger.
To do so, please create a port-forward
for the `jaeger-query` service:

```shell
kubectl port-forward -n keptn-system svc/jaeger-query 16686 
```

Afterwards, you can view the Jaeger UI in the browser at
[localhost:16686](http://localhost:16686).
There you should see the traces generated by the lifecycle operator,
which should look like this:

### Traces overview

![Screenshot of the traces overview in Jaeger](./assets/traces_overview.png)

### Trace details

![Screenshot of a trace in Jaeger](./assets/trace_detail.png)

In Prometheus, do a port forward to the prometheus service
inside your cluster (the exact name and namespace of the
prometheus service will depend on your Prometheus setup;
we are using the defaults that come with
the example of the Prometheus Operator tutorial).

```shell
kubectl -n monitoring port-forward svc/prometheus-k8s 9090
```

Afterwards, you can view the Prometheus UI in the browser at
[localhost:9090](http://localhost:9090).
There, in the
[Targets](http://localhost:9090/targets?search=) section,
you should see an entry for the otel-collector:

![Screenshot of a target in Prometheus](./assets/prometheus_targets.png)

Also, in the
[Graph](http://localhost:9090/graph?g0.expr=&g0.tab=1&g0.stacked=0&g0.show_exemplars=0&g0.range_input=1h)
section, you can retrieve metrics reported by Keptn
(all of the available metrics start with the `keptn` prefix):

![Screenshot of the auto-complete menu in a Prometheus query](./assets/metrics.png)

To view the exported metrics in Grafana,
we have provided dashboards
that have been automatically installed with this example.
To display them,
create a port-forward for the `grafana` service
in the `monitoring` namespace:

```shell
make port-forward-grafana
```

Now, you should be able to see it in the
[Grafana UI](http://localhost:3000/d/wlo2MpIVk/keptn-lifecycle-toolkit-metrics)
under `Dashboards > General`.

![Screenshot of a dashboard in Grafana](./assets/grafana_dashboard.png)