Bad certificates error on running logstash

[amolnater-qasource]

Kibana Build details:

VERSION: 8.15.1
BUILD: 76534
COMMIT: f66ec5b0ddd990d103489c47ca1bcb97dc50bc6b

Preconditions:

1. 8.15.1 Self-managed environment should be available.

Steps to reproduce:

1. Create certs using below command:

elasticsearch-certutil ca --pem
elasticsearch-certutil cert --name logstash --ca-cert C:\elk\elasticsearch\ca\ca.crt --ca-key C:\elk\elasticsearch\ca\ca.key --dns <public-dns> --ip <public-ip> --pem
elasticsearch-certutil cert --name client --ca-cert C:\elk\elasticsearch\ca\ca.crt --ca-key C:\elk\elasticsearch\ca\ca.key --dns <public-dns> --ip <public-ip> --pem

2. Convert logstash key to openssl pkcs8 -inform PEM -in logstash.key -topk8 -nocrypt -outform PEM -out logstash.pkcs8.key
3. Use elasticsearch/config/http_ca.crt as cacert and moved it to C:\elk\logstash\config\.
4. Update elastic-agent-pipeline.conf to:

input {
  elastic_agent {
    port => 5044
    ssl => true
    ssl_certificate_authorities => ["C:\elk\elasticsearch\ca\ca.crt"]
	ssl_certificate => "C:\elk\elasticsearch\logstash\logstash.crt"
	ssl_key => "C:\elk\elasticsearch\logstash\logstash.pkcs8.key"
    ssl_verify_mode => "force_peer"
  }
}
output {
  elasticsearch {
    hosts => "<elasticsearchhost>"
    api_key => "<api_key>"
    data_stream => true
    ssl => true
    cacert => "C:\elk\logstash\config\http_ca.crt"
  }
}

5. Update pipelines.yml to:

- pipeline.id: elastic-agent-pipeline
  path.config: "C:\elk\logstash\config\elastic-agent-pipeline.conf"  

6. Run logstash using: logstash -f C:\elk\logstash\config\elastic-agent-pipeline.conf.
7. Observe certificates error is visible.

Expected Result:
Logstash should run without any certificate errors.

Logs:
Logs.txt

Screenshot:

Feature:
elastic/kibana#75867

[elasticmachine]

Pinging @elastic/elastic-agent-control-plane (Team:Elastic-Agent-Control-Plane)

[amolnater-qasource]

@muskangulati-qasource Please review.

[muskangulati-qasource]

Secondary review for this ticket is done!

[cmacknz]

Logstash doesn't like something about the certs, transferring to Logstash as their more likely to have ideas about what the possible sources for the bad_certificate exception are here.