Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[O365_metrics] Add teams_user_activity datastream #12219

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ritalwar
Copy link
Contributor

@ritalwar ritalwar commented Jan 2, 2025

  • Enhancement

Proposed commit message

Add teams_user_activity datastream to O365_metrics package.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

How to test this PR locally

Related issues

Screenshots

@ritalwar ritalwar added Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations] Integration:o365_metrics Microsoft Office 365 Metrics labels Jan 2, 2025
@ritalwar ritalwar requested a review from lucian-ioan January 2, 2025 08:19
@ritalwar ritalwar self-assigned this Jan 2, 2025
@ritalwar ritalwar requested a review from a team as a code owner January 2, 2025 08:19
@andrewkroh andrewkroh added the enhancement New feature or request label Jan 2, 2025
@elastic-vault-github-plugin-prod

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

Copy link

Quality Gate failed Quality Gate failed

Failed conditions
50.0% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube

@elasticmachine
Copy link

💚 Build Succeeded

cc @ritalwar

required: false
show_user: false
default:
- o365.metrics.outlook.activity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rename tag.

value: "8.16.0"
- json:
field: teamsuseractivity
target_field: o365.metrics.teams.user.activity
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add ignore_failure: true here so the pipeline doesn't break.


### Teams User Activity

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggestion: Please add a one or two-line summary of what the dataset metrics are about.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type: integer
description: |
The count of other user actions within Teams.
- name: private_chat_messages.count
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As with the other datasets, kindly verify if the 2**31 value length is enough to hold the value received? If not enough, kindly choose a different field type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

long would be a safe or an unsigned type would be a safe choice in general.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

type: date
description: |
The date when the report data was last updated.
- name: date
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this different from the @timestamp field value ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, details in this comment.

- name: report
type: group
fields:
- name: period.day
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this value same as the period configuration value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can configure values such as D7, D30, etc., in the period, and it will display 7 and 30 accordingly.

ignore_missing: true
- rename:
field: o365.metrics.teams.user.activity.report_date
target_field: o365.metrics.teams.user.activity.report.date
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we do a explicit date handling based on type using the date processor?

source: |
String underscore(String s) {
String result = /[ -]/.matcher(s).replaceAll('_').toLowerCase();
return /[\ufeff]/.matcher(result).replaceAll('')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this bom char coming?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is a BOM character present in some fields, and I should update the pipeline comment to specify this as well.

String result = /[ -]/.matcher(s).replaceAll('_').toLowerCase();
return /[\ufeff]/.matcher(result).replaceAll('')
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clear trailing spaces

target_field: o365.metrics.teams.user.activity
- script:
lang: painless
description: Replace spaces and dashes in field names under o365.metrics.teams.user.activity.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no mention of bom removal?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will mention, thanks!

@@ -0,0 +1,83 @@
{
"o365": {
"metrics": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ritalwar - Query related to the metrics association. Is there a way the teams user activity metrics could be associated with the user-id or name to understand from where the request originates. If the user enables this specific data stream for user activity, how do they identify for which user this metrics is related to?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a limitation in API response itself?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @muthu-mps , this data stream is primarily designed to retrieve the number of Microsoft Teams users categorized by activity type. The activity types include team chat messages, private chat messages, calls, and meetings. Refer to the documentation here.

I'll add such links to the README for each dataset to prevent similar confusion.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When the API says user activity, I thought somehow they will link user to the activity type but this seems to be overall activity counts and not specific to a user.

Copy link
Contributor

@kcreddy kcreddy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ritalwar, is this data-stream going to be renamed to teams_user_activity_user_counts? I am working on teams_user_activity_user_detail here.

There was an agreement on creating data-streams viva_engage_device_usage_user_counts and viva_engage_groups_activity_group_details. I think we should continue the same norm here to avoid confusion for the users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request Integration:o365_metrics Microsoft Office 365 Metrics Team:Obs-InfraObs Label for the Observability Infrastructure Monitoring team [elastic/obs-infraobs-integrations]
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants