Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Development: Add more data for Telemetry #9345

Open
wants to merge 30 commits into
base: develop
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 23 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
848eb38
add more telemetry data
SimonEntholzer Sep 20, 2024
3a5d86f
disable on dev profile again
SimonEntholzer Sep 20, 2024
3829a34
improve coverage
SimonEntholzer Sep 20, 2024
e5c690a
simplify refactor and improve coverage
SimonEntholzer Sep 21, 2024
7c972b8
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Sep 23, 2024
8cb8520
improved service, incorporating feedback
SimonEntholzer Sep 26, 2024
8988d00
undo of disable disable
SimonEntholzer Sep 26, 2024
19c2db1
removed eureka querying until we have a better approach
SimonEntholzer Sep 28, 2024
97fa018
removed now unused EurekaClientService
SimonEntholzer Sep 28, 2024
c4091b8
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Sep 28, 2024
3e07931
Merge remote-tracking branch 'refs/remotes/origin/develop' into featu…
SimonEntholzer Sep 28, 2024
eb973ce
resolved merge conflict
SimonEntholzer Sep 28, 2024
389aba7
moved exception handling inside async function and added additional l…
SimonEntholzer Sep 29, 2024
fb396a2
re-added eurekaClientService, and schedule telemetry task 2 minutes a…
SimonEntholzer Oct 1, 2024
e9ec333
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 2, 2024
58f0212
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 8, 2024
5d35b48
remove Async as task is scheduled now
SimonEntholzer Oct 8, 2024
c28f7fb
increase delay
SimonEntholzer Oct 8, 2024
bf31151
add comment
SimonEntholzer Oct 8, 2024
8fea6c1
remove delay in tests
SimonEntholzer Oct 8, 2024
ea0c9c3
put parameters in constructor
SimonEntholzer Oct 9, 2024
49423d7
removed unused property
SimonEntholzer Oct 9, 2024
f18e15f
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 12, 2024
e51fa7c
removed multi node and build agent telemetry data for now
SimonEntholzer Oct 12, 2024
7efc0ac
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 12, 2024
3b7f759
Update src/main/java/de/tum/cit/aet/artemis/core/service/telemetry/Te…
SimonEntholzer Oct 12, 2024
655c571
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 13, 2024
4d16985
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 14, 2024
5d9f093
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 14, 2024
3de091e
Merge branch 'develop' into feature/telemetry/add-additional-fields
SimonEntholzer Oct 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

This file was deleted.

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
package de.tum.cit.aet.artemis.core.service.telemetry;

import static de.tum.cit.aet.artemis.core.config.Constants.PROFILE_SCHEDULING;

import java.net.URI;
import java.nio.charset.StandardCharsets;
import java.util.Base64;
import java.util.Collections;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Profile;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.HttpMethod;
import org.springframework.http.MediaType;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;

@Profile(PROFILE_SCHEDULING)
@Service
public class EurekaClientService {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

private static final Logger log = LoggerFactory.getLogger(EurekaClientService.class);

@Value("${eureka.client.service-url.defaultZone}")
private String eurekaServiceUrl;

private final RestTemplate restTemplate;

public EurekaClientService(RestTemplate restTemplate) {
this.restTemplate = restTemplate;
}

/**
* Retrieves the number of Artemis application instances registered with the Eureka server.
* <p>
* This method makes an HTTP GET request to the Eureka server's `/api/eureka/applications` endpoint to
* retrieve a list of all registered applications. It then filters out the Artemis application
* and returns the number of instances associated with it. If any error occurs during the request
* (e.g., network issues, parsing issues, invalid URI), the method returns 1 as the default value.
*
* @return the number of Artemis application instances, or 1 if an error occurs.
*/
public long getNumberOfReplicas() {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
try {
var eurekaURI = new URI(eurekaServiceUrl);
HttpHeaders headers = createHeaders(eurekaURI.getUserInfo());
HttpEntity<String> request = new HttpEntity<>(headers);
var requestUrl = eurekaURI.getScheme() + "://" + eurekaURI.getAuthority() + "/api/eureka/applications";

ResponseEntity<String> response = restTemplate.exchange(requestUrl, HttpMethod.GET, request, String.class);

for (JsonNode application : new ObjectMapper().readTree(response.getBody()).get("applications")) {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
if (application.get("name").asText().equals("ARTEMIS")) {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
return application.get("instances").size();
}
}
}
catch (Exception e) {
log.warn("Error while trying to retrieve number of replicas.");
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

return 1;
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Creates HTTP headers with Basic Authentication and JSON content type.
*
* @param auth the user credentials in the format "username:password" to be encoded and included in the Authorization header.
* @return HttpHeaders with Basic Authentication and JSON content types.
*/
private HttpHeaders createHeaders(String auth) {
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
headers.setAccept(Collections.singletonList(MediaType.APPLICATION_JSON));

byte[] encodedAuth = Base64.getEncoder().encode(auth.getBytes(StandardCharsets.US_ASCII));
String authHeader = "Basic " + new String(encodedAuth);

headers.set("Authorization", authHeader);
return headers;
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
package de.tum.cit.aet.artemis.core.service.telemetry;

import static de.tum.cit.aet.artemis.core.config.Constants.PROFILE_SCHEDULING;

import java.util.Arrays;
import java.util.List;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Profile;
import org.springframework.core.env.Environment;
import org.springframework.http.HttpEntity;
import org.springframework.http.HttpHeaders;
import org.springframework.http.MediaType;
import org.springframework.stereotype.Service;
import org.springframework.web.client.RestTemplate;

import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;

import de.tum.cit.aet.artemis.core.service.ProfileService;

@Service
@Profile(PROFILE_SCHEDULING)
public class TelemetrySendingService {

private static final Logger log = LoggerFactory.getLogger(TelemetrySendingService.class);

@JsonInclude(JsonInclude.Include.NON_EMPTY)
public record TelemetryData(String version, String serverUrl, String operator, List<String> profiles, boolean isProductionInstance, boolean isTestServer, String dataSource,
boolean isMultiNode, long numberOfNodes, long buildAgentCount, String contact, String adminName) {
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

private final Environment env;

private final RestTemplate restTemplate;

private final ProfileService profileService;

private final EurekaClientService eurekaClientService;

public TelemetrySendingService(Environment env, RestTemplate restTemplate, ProfileService profileService, EurekaClientService eurekaClientService) {
this.env = env;
this.restTemplate = restTemplate;
this.profileService = profileService;
this.eurekaClientService = eurekaClientService;
}

@Value("${artemis.version}")
private String version;

@Value("${server.url}")
private String serverUrl;

@Value("${info.operatorName}")
private String operator;

@Value("${info.operatorAdminName}")
private String operatorAdminName;

@Value("${info.contact}")
private String operatorContact;
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@Value("${artemis.telemetry.destination}")
private String destination;

@Value("${spring.datasource.url}")
private String datasourceUrl;
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

@Value("${artemis.continuous-integration.concurrent-build-size:0}")
private long buildAgentCount;

@Value("${info.test-server:false}")
private boolean isTestServer;

/**
* Sends telemetry data to a specified destination via an HTTP POST request asynchronously.
* The telemetry includes information about the application version, environment, data source,
* and optionally, administrator details. If Eureka is enabled, the number of registered
* instances is also included.
*
* <p>
* The method constructs the telemetry data object, converts it to JSON, and sends it to a
* telemetry collection server. The request is sent asynchronously due to the {@code @Async} annotation.
*
* @param eurekaEnabled a flag indicating whether Eureka is enabled. If {@code true},
* the method retrieves the number of instances registered with Eureka.
* @param sendAdminDetails a flag indicating whether to include administrator details in the
* telemetry data (such as contact information and admin name).
*/
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
public void sendTelemetryByPostRequest(boolean eurekaEnabled, boolean sendAdminDetails) {
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

long numberOfNodes = 1;

if (eurekaEnabled) {
log.info("Querying other instances from Eureka...");
numberOfNodes = eurekaClientService.getNumberOfReplicas();
}

try {
String telemetryJson = new ObjectMapper().writer().withDefaultPrettyPrinter().writeValueAsString(buildTelemetryData(sendAdminDetails, eurekaEnabled, numberOfNodes));
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.APPLICATION_JSON);
HttpEntity<String> requestEntity = new HttpEntity<>(telemetryJson, headers);

log.info("Sending telemetry to {}", destination);
var response = restTemplate.postForEntity(destination + "/api/telemetry", requestEntity, String.class);
log.info("Successfully sent telemetry data. {}", response.getBody());
}
catch (JsonProcessingException e) {
log.warn("JsonProcessingException in sendTelemetry.", e);
}
catch (Exception e) {
log.warn("Exception in sendTelemetry, with dst URI: {}", destination, e);
}
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}

/**
* Retrieves telemetry data for the current system configuration, including details
* about the active profiles, data source type, and optionally admin contact details.
*
* @param sendAdminDetails whether to include admin contact information in the telemetry data
* @param isMultiNode whether the application runs in multi node or single node mode
* @param numberOfNodes the number of nodes which are part of the cluster
* @return an instance of {@link TelemetryData} containing the gathered telemetry information
*/
private TelemetryData buildTelemetryData(boolean sendAdminDetails, boolean isMultiNode, long numberOfNodes) {
TelemetryData telemetryData;
var dataSource = datasourceUrl.startsWith("jdbc:mysql") ? "mysql" : "postgresql";
List<String> activeProfiles = Arrays.asList(env.getActiveProfiles());
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved

String contact = null;
String adminName = null;
if (sendAdminDetails) {
contact = operatorContact;
adminName = operatorAdminName;
}
telemetryData = new TelemetryData(version, serverUrl, operator, activeProfiles, profileService.isProductionActive(), isTestServer, dataSource, isMultiNode, numberOfNodes,
buildAgentCount, contact, adminName);
return telemetryData;
SimonEntholzer marked this conversation as resolved.
Show resolved Hide resolved
}
}
Loading
Loading