XMLWordPrintable

    Details

    • Type: Bug
    • Status: Open
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Logging
    • Labels:
      None
    • Environment:

      All envirments 

      Description

      To whom it may concern,

       

      As far as we know, there are 5 microservices which make up the X-Ray Architecture as shown below

       

      • Indexing
      • Persist
      • Analysis
      • Server
      • Router

       

      Each of the above component steamed to Splunk, each have it own sourcetype, however the data is inconsistent, which is difficult to extract the right attribute as shown below, unlike the artifactory logs where all logs are consistent and we are able to extract a single field and map to the value.

       

      As you can see from the below logs, the first and second line are complete different, hence extracting a field is impossible.

      Server Logs:

      created by jfrog.com/jfrog-go-commons/v7/pkg/metrics.(*collector).collect /go/pkg/mod/jfrog.com/jfrog-go-commons/v7@v7.19.0-m019/pkg/metrics/collector.go:136 +0x1c6

      2021-09-14T05:29:52.164Z [33m[jfxr ][0m [34m[INFO ][0m [a286e28f21a5e437] [alert_producer:139 ] [main ] Alerts created for Workflow: scan, Subject: maturity-instructions-test:master_97, Source: prod-Artifactory/ubank-docker/maturity-instructions-test/master_97/ in 1.639581638s. transaction cache stats: (WatchByName: total: 1, hits: 0, hit ratio: 0%) (PolicyByName: total: 1, hits: 0, hit ratio: 0%) (PublicVulnerabilitySeverity: total: 79, hits: 27, hit ratio: 34%) (PublicVulnerabilityInfo: total: 25, hits: 0, hit ratio: 0%)

       

      Persist Logs:

      2021-09-14T06:20:16.629Z [33m[jfxps][0m [34m[INFO ][0m [522d2a918322b79e] [storage_service:245 ] [main ] Persisting component graph docker://bulk-fetch-customer-info:snapshot-db9ff97de70a6d20196e359c4fc733657a417f9c to graph db

      2021-09-14T06:20:14.466Z [33m[jfxps][0m [34m[INFO ][0m [522d2a918322b79e] [persist_worker:99 ] [main ] Persist worker id 3 is processing message from persist --> /bulk-fetch-customer-info/snapshot-db9ff97de70a6d20196e359c4fc733657a417f9c/manifest.json

       

       

      From the above logs, it will best to split each field using a | i.e.

      2021-09-14T06:20:14.466Z |[33m[jfxps]|[0m [34m[INFO ]|[0m [522d2a918322b79e]| [persist_worker:99 ] | [main ] *|Persist worker id 3 is processing message from persist|* --> |/bulk-fetch-customer-info/snapshot-db9ff97de70a6d20196e359c4fc733657a417f9c/manifest.json

       

      The above format will help us to extract the field after the number |.

       

      In short, are you able to modify the X-Ray logs so it can be inline with Artifactory logs format?

       

      Thank you in advance

       

      Fadi

        Attachments

          Activity

              People

              Assignee:
              Unassigned
              Reporter:
              fkaba81 Fadi Kaba
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated: