Update dataset in packages

[ruflin]

For now, the packages have old and new fields inside. As soon as Kibana and Agent are shipped, the old definition can be removed. See also elastic/package-storage#56

[ruflin]

@mtojek For the dashboards I did a find / replace. I wonder if this catches all the occurences.

[mtojek]

Reviewing

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #64 updated]

• Start Time: 2020-06-09T14:08:15.301+0000

• Duration: 3 min 54 sec

[mtojek]

A couple of observations:

1. Do you think we can remove stream.* fields now? No dashboard or ingest pipeline will be using them. Maybe we can limit number of pushes to integrations and package-storage repository if the agent supports temporarily both.
2. Thanks for adjusting the dev/import-beats/fields_base_fields.go. I think there is small find & replace needed in kibana.go (import-beats).

[mtojek]

I left some observations.

[ruflin]

I removed the stream.* fields now as the agent supports both at the moment. Also modified the kibana.go, please have a look again.

[mtojek]

Thanks for changes in import-beats. If the CI goes green, feel free to merge.

[mtojek]

[2020-06-09T13:51:49.975Z] 2020/06/09 13:51:49 validating required fields failed (datasetPath: /var/lib/jenkins/workspace/Beats_integrations_PR-64/src/github.com/elastic/integrations/build/public/package/aws/0.1.0/dataset/billing): finding field failed (searchedName: stream.type): field 'stream.type' not found

I think you need to bump up the dependency on the package-registry.

[mtojek]

CI failed.