第四步:运行后端应用程序
下一步是运行后端应用程序。为此,请
- 创建 API 密钥以对后端应用程序进行身份验证。
- 在 Linux 或 Kubernetes 上运行应用程序。
收集器(collector)和符号化器(symbolizer)都需要向 Elasticsearch 进行身份验证以处理性能分析数据。为此,您需要为每个应用程序创建一个 API 密钥。
请参阅创建 API 密钥以使用 Kibana 创建 API 密钥。选择一个 User API key(用户 API 密钥),并在 Control security privileges(控制安全权限) 下分配以下权限:
{
"profiling": {
"cluster": [
"monitor"
],
"indices": [
{
"names": [
"profiling-*"
],
"privileges": [
"read",
"write"
]
}
]
}
}
保存 API 密钥的“已编码 (Encoded)”版本,因为运行 Universal Profiling 后端需要用到它们。继续阅读在 Linux 上运行或在 Kubernetes 上运行,了解有关运行后端应用程序的信息。
在 Linux 上运行后端应用程序之前,我们建议创建配置文件来管理应用程序。虽然也支持 CLI 标志,但它们可能会导致后端应用程序的管理变得更加复杂。
使用以下选项之一安装后端应用程序
- 操作系统软件包 (DEB/RPM)
- OCI 容器
- 二进制文件:使用您选择的配置管理系统(Ansible、Puppet、Chef、Salt 等)进行编排
配置文件采用 YAML 格式,由两个顶级部分组成:“application”部分和“output”部分。
“application”部分包含后端应用程序的配置,“output”部分包含用于连接到要读取和发送数据的目标位置的配置。“application”部分以二进制文件的名称命名。“output”部分目前仅支持 Elasticsearch。
配置文件从以下默认位置读取:
- 收集器 (Collector):
/etc/Elastic/universal-profiling/pf-elastic-collector.yml - 符号化器 (Symbolizer):
/etc/Elastic/universal-profiling/pf-elastic-symbolizer.yml
您可以在运行应用程序时使用 -c 标志来自定义配置文件的位置。
为简单起见,我们在下面的示例中将使用默认位置。我们还展示了默认的应用程序设置;您可以参考 YAML 中的注释来了解如何自定义它们。
将下面代码片段的内容复制到 /etc/Elastic/universal-profiling/pf-elastic-collector.yml 文件中。
使用您选择的密钥令牌自定义 pf-elastic-collector.auth.secret_token 的内容。Universal Profiling Agent 将使用此令牌向收集器进行身份验证;您不能使用空字符串作为令牌。如果您想使用 TLS 保护收集器的端点,请调整 ssl 部分。
自定义 output.elasticsearch 部分的内容,分别使用 Elasticsearch 端点和API 密钥来设置 hosts 和 api_key 值。根据需要调整 protocol 值和其他与 TLS 相关的设置。
收集器配置文件
pf-elastic-collector:
# Defines the host and port the server is listening on.
host: "0.0.0.0:8260"
# Verbose log output option.
#verbose: true
# Configure metrics exposition. Both expvar and Prometheus formats are supported. Both can be
# configured at the same time. By default, no metrics are exposed.
# 'prometheus_host' can only be configured with a 'host:port' pair.
# 'expvar_host' can be configured either with a 'host:port' pair or with a Unix Domain Socket path (with a 'unix://' prefix).
# When host:port is used, an HTTP server is exposed. The server does not support TLS.
# An empty value disables metrics exposition for the corresponding format.
#metrics:
# prometheus_host: 'localhost:9090'
# expvar_host: unix:///tmp/collector-metrics.sock
# Define the suggested upper limit of memory that pf-elastic-collector should apply. Using a lower
# amount of memory might trigger garbage collection more often.
#memory_limit: 500M
# Agent authorization configuration. If no methods are defined, all requests will be allowed.
auth:
# Define a shared secret token for authorizing agents.
secret_token: ""
# Controls storage of Universal Profiling agent metrics/metadata to the customer's cluster and to a
# cluster controlled by Elastic. By default, the full set of metrics and metadata is written to
# an Elastic-controlled cluster, and a subset of metrics and metadata to the customer
# cluster. These are used to monitor agent health and debug/resolve issues.
agent_metrics:
# Do not write Universal Profiling agent metrics/metadata to a centralized (non-customer controlled)
# cluster. This does not affect writing metrics/metadata to the customer cluster.
#disable: false
# Write full set of Universal Profiling agent metrics to the customer ES cluster. If false, which
# is the default, only a limited set of CPU usage and I/O metrics will be written.
#write_all: false
# Enable secure communication between pf-host-agent and pf-elastic-collector.
ssl:
enabled: false
# Path to file containing the certificate for server authentication.
# Needs to be configured when ssl is enabled.
#certificate: ''
# Path to file containing server certificate key.
# Needs to be configured when ssl is enabled.
#key: ''
# Optional configuration options for ssl communication.
# Passphrase for decrypting the Certificate Key.
# It is recommended to use the provided keystore instead of entering the passphrase in plain text.
#key_passphrase: ''
# List of supported/valid protocol versions. By default TLS versions 1.3 is enabled.
#supported_protocols: [TLSv1.3]
# Configure cipher suites to be used for SSL connections.
# Note that cipher suites are not configurable for TLS 1.3.
#cipher_suites: []
# Configure curve types for ECDHE based cipher suites.
#curve_types: []
#================================ Outputs =================================
# Configure the output to use when sending the data collected by pf-elastic-collector.
#-------------------------- Elasticsearch output --------------------------
output:
elasticsearch:
# Array of hosts to connect to.
# Scheme and port can be left out and will be set to the default (`http` and `9200`).
# In case you specify an additional path, the scheme is required: `https://:9200/path`.
# IPv6 addresses should always be defined as: `https://[2001:db8::1]:9200`.
hosts: ["localhost:9200"]
# Set gzip compression level.
#compression_level: 0
# Protocol - either `http` (default) or `https`.
protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
# Optional HTTP Path.
#path: "/elasticsearch"
# Proxy server url.
#proxy_url: http://proxy:3128
# The number of times a particular Elasticsearch index operation is attempted. If
# the indexing operation doesn't succeed after this many retries, the events are
# dropped. The default is 3.
#max_retries: 3
# Enable custom SSL settings. Set to false to ignore custom SSL settings for secure communication.
#ssl.enabled: true
# Optional SSL configuration options. SSL is off by default, change the `protocol` option if you want to enable `https`.
#
# Control the verification of Elasticsearch certificates. Valid values are:
# * full, which verifies that the provided certificate is signed by a trusted
# authority (CA) and also verifies that the server's hostname (or IP address)
# matches the names identified within the certificate.
# * strict, which verifies that the provided certificate is signed by a trusted
# authority (CA) and also verifies that the server's hostname (or IP address)
# matches the names identified within the certificate. If the Subject Alternative
# Name is empty, it returns an error.
# * certificate, which verifies that the provided certificate is signed by a
# trusted authority (CA), but does not perform any hostname verification.
# * none, which performs no verification of the server's certificate. This
# mode disables many of the security benefits of SSL/TLS and should only be used
# after very careful consideration. It is primarily intended as a temporary
# diagnostic mechanism when attempting to resolve TLS errors; its use in
# production environments is strongly discouraged.
#ssl.verification_mode: full
# List of supported/valid TLS versions. By default all TLS versions 1.0 up to
# 1.2 are enabled.
#ssl.supported_protocols: [TLSv1.0, TLSv1.1, TLSv1.2]
# List of root certificates for HTTPS server verifications.
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication.
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
# Optional passphrase for decrypting the Certificate Key.
# It is recommended to use the provided keystore instead of entering the passphrase in plain text.
#ssl.key_passphrase: ''
# Configure cipher suites to be used for SSL connections.
#ssl.cipher_suites: []
# Configure curve types for ECDHE based cipher suites.
#ssl.curve_types: []
# Configure what types of renegotiation are supported. Valid options are
# never, once, and freely. Default is never.
#ssl.renegotiation: never
将下面代码片段的内容复制到 /etc/Elastic/universal-profiling/pf-elastic-symbolizer.yml 文件中。
您无需自定义 pf-elastic-symbolizer 部分中的任何值。如果您想使用 TLS 保护符号化器的端点,请调整 ssl 部分。
自定义 output.elasticsearch 部分的内容,分别使用 Elasticsearch 端点和API 密钥来设置 hosts 和 api_key 值。根据需要调整 protocol 值和其他与 TLS 相关的设置。
符号化器配置文件
pf-elastic-symbolizer:
# Defines the host and port the server is listening on.
host: "0.0.0.0:8240"
# Endpoint for the service to connect to and query for software packages.
# Do not set this value unless you are running a local instance of the debug symbols mirror.
endpoint: ""
# Verbose log output option. (default: false)
#verbose: true
# Configure metrics exposition. Both expvar and Prometheus formats are supported. Both can be
# configured at the same time. By default, no metrics are exposed.
# 'prometheus_host' can only be configured with a 'host:port' pair.
# 'expvar_host' can be configured either with a 'host:port' pair or with a Unix Domain Socket path (with a 'unix://' prefix).
# When host:port is used, an HTTP server is exposed. The server does not support TLS.
# An empty value disables metrics exposition for the corresponding format.
#metrics:
# prometheus_host: 'localhost:9090'
# expvar_host: unix:///tmp/collector-metrics.sock
# Define the suggested upper limit of memory that pf-elastic-symbolizer should apply. Using a lower
# amount of memory might trigger garbage collection more often. (default: 200MB)
#memory_limit: 500M
# Enable secure communication between symbtool and pf-elastic-symbolizer.
ssl:
enabled: false
# Path to file containing the certificate for server authentication.
# Needs to be configured when ssl is enabled.
#certificate: ''
# Path to file containing server certificate key.
# Needs to be configured when ssl is enabled.
#key: ''
# Optional configuration options for ssl communication.
# Passphrase for decrypting the Certificate Key.
# It is recommended to use the provided keystore instead of entering the passphrase in plain text.
#key_passphrase: ''
# List of supported/valid protocol versions. By default TLS versions 1.1 up to 1.3 are enabled.
#supported_protocols: [TLSv1.1, TLSv1.2, TLSv1.3]
# Configure cipher suites to be used for SSL connections.
# Note that cipher suites are not configurable for TLS 1.3.
#cipher_suites: []
# Configure curve types for ECDHE based cipher suites.
#curve_types: []
#================================ Outputs =================================
# Configure the output to use when sending the data collected by pf-elastic-symbolizer.
#-------------------------- Elasticsearch output --------------------------
output:
elasticsearch:
# Array of hosts to connect to.
# Scheme and port can be left out and will be set to the default (`http` and `9200`).
# In case you specify an additional path, the scheme is required: `https://:9200/path`.
# IPv6 addresses should always be defined as: `https://[2001:db8::1]:9200`.
hosts: ["localhost:9200"]
# Set gzip compression level.
#compression_level: 0
# Protocol - either `http` (default) or `https`.
protocol: "https"
# Authentication credentials - either API key or username/password.
#api_key: "id:api_key"
# Optional HTTP Path.
#path: "/elasticsearch"
# Proxy server url.
#proxy_url: http://proxy:3128
# The number of times a particular Elasticsearch index operation is attempted. If
# the indexing operation doesn't succeed after this many retries, the events are
# dropped. The default is 3.
#max_retries: 3
# Enable custom SSL settings. Set to false to ignore custom SSL settings for secure communication.
#ssl.enabled: true
# Optional SSL configuration options. SSL is off by default, change the `protocol` option if you want to enable `https`.
#
# Control the verification of Elasticsearch certificates. Valid values are:
# * full, which verifies that the provided certificate is signed by a trusted
# authority (CA) and also verifies that the server's hostname (or IP address)
# matches the names identified within the certificate.
# * strict, which verifies that the provided certificate is signed by a trusted
# authority (CA) and also verifies that the server's hostname (or IP address)
# matches the names identified within the certificate. If the Subject Alternative
# Name is empty, it returns an error.
# * certificate, which verifies that the provided certificate is signed by a
# trusted authority (CA), but does not perform any hostname verification.
# * none, which performs no verification of the server's certificate. This
# mode disables many of the security benefits of SSL/TLS and should only be used
# after very careful consideration. It is primarily intended as a temporary
# diagnostic mechanism when attempting to resolve TLS errors; its use in
# production environments is strongly discouraged.
#ssl.verification_mode: full
# List of supported/valid TLS versions. By default all TLS versions 1.0 up to
# 1.2 are enabled.
#ssl.supported_protocols: [TLSv1.0, TLSv1.1, TLSv1.2]
# List of root certificates for HTTPS server verifications.
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication.
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
# Optional passphrase for decrypting the Certificate Key.
# It is recommended to use the provided keystore instead of entering the passphrase in plain text.
#ssl.key_passphrase: ''
# Configure cipher suites to be used for SSL connections.
#ssl.cipher_suites: []
# Configure curve types for ECDHE based cipher suites.
#ssl.curve_types: []
# Configure what types of renegotiation are supported. Valid options are
# never, once, and freely. Default is never.
#ssl.renegotiation: never
按照以下步骤使用操作系统软件包安装后端。
配置 APT 仓库
wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add - sudo apt-get install apt-transport-https echo "deb https://artifacts.elastic.co/packages/9.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-9.x.list安装软件包
sudo apt update sudo apt install -y pf-elastic-collector pf-elastic-symbolizer
对于 RPM 软件包,请配置 YUM 仓库并安装软件包
下载并安装公共签名密钥
sudo rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch在您的
/etc/yum.repos.d/目录下创建一个扩展名为.repo的文件(例如elastic.repo),并添加以下内容[elastic-9.x] name=Elastic repository for 9.x packages baseurl=https://artifacts.elastic.co/packages/9.x/yum gpgcheck=1 gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch enabled=1 autorefresh=1 type=rpm-md运行以下命令安装后端服务:
sudo yum update sudo yum install -y pf-elastic-collector pf-elastic-symbolizer
安装软件包后,启用并启动 systemd 服务:
sudo systemctl enable pf-elastic-collector
sudo systemctl start pf-elastic-collector
sudo systemctl enable pf-elastic-symbolizer
sudo systemctl start pf-elastic-symbolizer
现在,您可以检查服务的日志以发现任何问题:
sudo journalctl -xu pf-elastic-collector
sudo journalctl -xu pf-elastic-symbolizer
有关排查日志中可能出现的错误的更多信息,请参阅排查 Universal Profiling 后端故障。
我们在 Elastic 镜像仓库中提供 OCI 镜像,以便在容器中运行后端服务。这些镜像是多平台的,因此它们既适用于 x86_64 架构,也适用于 ARM64 架构。
将配置文件放置在系统中后,您可以使用以下命令运行容器(示例命令使用 Docker,但任何 OCI 运行时都可以):
收集器
docker run -d --name pf-elastic-collector -p 8260:8260 -v /etc/Elastic/universal-profiling/pf-elastic-collector.yml:/pf-elastic-collector.yml:ro \ docker.elastic.co/observability/profiling-collector:{version} -c /pf-elastic-collector.yml符号化器 (Symbolizer)
docker run -d --name pf-elastic-symbolizer -p 8240:8240 -v /etc/Elastic/universal-profiling/pf-elastic-symbolizer.yml:/pf-elastic-symbolizer.yml:ro \ docker.elastic.co/observability/profiling-symbolizer:{version} -c /pf-elastic-symbolizer.yml
通过上述命令,后端容器将分别在宿主机端口 8260 和 8240 上提供 HTTP 端点服务。我们提供了 -v 标志将配置文件挂载到容器中,然后使用 -c 标志指示应用程序从挂载的路径读取配置文件。
容器进程将在后台运行,您可以使用 docker logs <container_name> 检查日志,例如:
docker logs pf-elastic-collector
docker logs pf-elastic-symbolizer
下载并解压适合您平台的二进制文件:
对于 x86_64
wget -O- "https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-collector-9.4.4-linux-x86_64.tar.gz" | tar xzf - wget -O- "https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-symbolizer-9.4.4-linux-x86_64.tar.gz" | tar xzf -对于 ARM64
wget -O- "https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-collector-9.4.4-linux-arm64.tar.gz" | tar xzf - wget -O- "https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-symbolizer-9.4.4-linux-arm64.tar.gz" | tar xzf -将
pf-elastic-collector和pf-elastic-symbolizer二进制文件复制到机器PATH环境变量中的目录。运行后端应用程序进程,指示它们读取先前创建的配置文件。
pf-elastic-collector -c /etc/Elastic/universal-profiling/pf-elastic-collector.yml pf-elastic-symbolizer -c /etc/Elastic/universal-profiling/pf-elastic-symbolizer.yml
如果要自定义传递给二进制文件的配置选项,可以使用命令行标志。所有覆盖项都使用 -E 标志指定。例如,如果要覆盖 pf-elastic-collector 应用程序的 host 值,可以使用如下 -E pf-elastic-collector.host 标志:
pf-elastic-collector -c /etc/Elastic/universal-profiling/pf-elastic-collector.yml -E pf-elastic-collector.host=0.0.0.0:8844
在上一个示例中,我们配置收集器监听端口 8844 上的所有网络接口,而不是 YAML 配置文件中包含的 8260 值。
您可以使用 -E 标志覆盖配置文件中包含的任何值,只要您在命令行标志上指定完整的 YAML 路径即可。为了更简单的编排,我们建议坚持使用配置文件。
相同的配置覆盖和建议也适用于 pf-elastic-symbolizer 二进制文件。
我们提供 Helm chart,用于在 Kubernetes 上部署后端服务。
要安装后端服务,您需要将 Elastic Helm 仓库添加到您的 Helm 安装中,然后安装 chart。
我们建议创建一个 values.yaml 文件来定义 chart 的 Kubernetes 特定选项。如果您想使用 chart 提供的默认值,则无需为每个 chart 创建 values.yaml 文件。对于应用程序的配置,您可以复用“创建配置文件”中详细说明的配置文件,并将其作为 values 文件传递给 Helm(使用 --values 或 -f 标志),或者将其复制到 values.yaml 文件中。
在下面的示例中,我们不对 Kubernetes 配置进行任何修改,因此我们将使用 chart 提供的默认值。
安装并更新 Elastic Helm 仓库
helm repo add elastic https://helm.elastic.co helm repo update elastic安装 chart(我们使用
universal-profiling命名空间,但您可以随意自定义)helm install --create-namespace -n universal-profiling collector elastic/profiling-collector -f /etc/Elastic/universal-profiling/pf-elastic-collector.yml helm install --create-namespace -n universal-profiling symbolizer elastic/profiling-symbolizer -f /etc/Elastic/universal-profiling/pf-elastic-symbolizer.yml按照
helm install命令输出中列出的步骤,检查 pod 是否正在运行并读取其日志。
在先前的示例中,我们使用 chart 的默认值来配置 Kubernetes 资源。这些不包括创建 Ingress 资源。如果您想将服务暴露给 Kubernetes 集群外部的 Universal Profiling Agent 和 symbtool 部署,您需要设置每个 chart 的 ingress 部分。
继续阅读第 5 步:后续步骤。