Skip to content

Conversation

alexandraBara
Copy link
Collaborator

Collecting system logs from /var/log/syslog*. Collection only, this plugin does not have an analyzer part yet.
Sample run:

(venv) (base) alexbara@pp-128-a5-1:~/node-scraper$ node-scraper run-plugins SyslogPlugin
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Log path: ./scraper_logs_pp_128_a5_1_2025_08_18-02_07_26_PM
  2025-08-18 14:07:26 CDT       INFO               nodescraper | System Name: <>
  2025-08-18 14:07:26 CDT       INFO               nodescraper | System SKU: None
  2025-08-18 14:07:26 CDT       INFO               nodescraper | System Platform: None
  2025-08-18 14:07:26 CDT       INFO               nodescraper | System location: SystemLocation.LOCAL
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Initializing connection manager for InBandConnectionManager with default args
  2025-08-18 14:07:26 CDT       INFO               nodescraper | --------------------------------------------------
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Running plugin SyslogPlugin
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Initializing connection: InBandConnectionManager
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Using local shell
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Checking OS family
  2025-08-18 14:07:26 CDT       INFO               nodescraper | OS Family: LINUX
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Running data collector: SyslogCollector
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Collected syslog log: rotated_syslog.log
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Collected syslog log: rotated_syslog.1.log
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Collected syslog log: rotated_syslog.2.gz.log
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Collected syslog log: rotated_syslog.3.gz.log
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Collected syslog log: rotated_syslog.4.gz.log
  2025-08-18 14:07:26 CDT       INFO               nodescraper | (SyslogPlugin) Syslog data collected
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Closing connections
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Running result collators
  2025-08-18 14:07:26 CDT       INFO               nodescraper | Running TableSummary result collator
  2025-08-18 14:07:26 CDT       INFO               nodescraper |

+-------------------------+--------+-----------------------------+
|  Connection              | Status | Message                     |
+-------------------------+--------+-----------------------------+
|  InBandConnectionManager | UNSET  | task completed successfully |
+-------------------------+--------+-----------------------------+

+--------------+--------+-------------------------------------+
|  Plugin       | Status | Message                             |
+--------------+--------+-------------------------------------+
|  SyslogPlugin | OK     | Plugin tasks completed successfully |
+--------------+--------+-------------------------------------+

  2025-08-18 14:07:26 CDT       INFO               nodescraper | Data written to csv file: ./scraper_logs_pp_128_a5_1_2025_08_18-02_07_26_PM/nodescraper.csv

class SyslogData(DataModel):
"""Data model for in band syslog logs"""

syslog_logs: int = 0
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@landrews-amd should i be keeping track of the files collected rather than the number of logs collected? (similar to the dmesg PR?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should include the log data itself here.

Comment on lines +45 to +47
def _shell_quote(self, s: str) -> str:
"""single-quote fix."""
return "'" + s.replace("'", "'\"'\"'") + "'"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be put into utils.py since it is being used in a few different places.

Comment on lines +49 to +71
def _nice_syslog_name(self, path: str) -> str:
"""Map path to filename
Args:
path (str): file path
Returns:
str: new local filename
"""
prefix = "rotated_"
base = path.rstrip("/").rsplit("/", 1)[-1]

if base == "syslog":
return f"{prefix}syslog.log"

m = re.fullmatch(r"syslog\.(\d+)\.gz", base)
if m:
return f"{prefix}syslog.{m.group(1)}.gz.log"

m = re.fullmatch(r"syslog\.(\d+)", base)
if m:
return f"{prefix}syslog.{m.group(1)}.log"

middle = base[:-3] if base.endswith(".gz") else base
return f"{prefix}{middle}.log"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may also be good to make more generic and move to utils since the same logic is used for the dmesg files.

Comment on lines +95 to +100
self.result.artifacts.append(
TextFileArtifact(filename=fname, contents=res.stdout)
)
collected_logs.append(
{"path": p, "as": fname, "bytes": len(res.stdout.encode("utf-8", "ignore"))}
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it will be better to store the artifacts as part of the data model rather than in artifacts in this case. This more closely aligns with the pattern of other collectors and will allow the logs to be analyzed in the future if need be.

class SyslogData(DataModel):
"""Data model for in band syslog logs"""

syslog_logs: int = 0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we should include the log data itself here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants