112 lines
3.5 KiB
Markdown
112 lines
3.5 KiB
Markdown
# Logar
|
|
|
|
Logar is a simple log archive tool to manage HTTP log files.
|
|
|
|
|
|
## LICENSE
|
|
|
|
Logar is released under the GNU AGPLv3+ license. Enjoy!
|
|
|
|
Authors:
|
|
* Christian Pierre MOMON <christian@momon.org>
|
|
|
|
|
|
### Logo
|
|
|
|
Author: Christian Pierre MOMON <christian@momon.org>
|
|
|
|
Based on https://publicdomainvectors.org/en/free-clipart/Wood-log-clip-art/45623.html
|
|
|
|
License: Public Domain.
|
|
|
|
### Other stuff
|
|
|
|
License of other stuff:
|
|
* other: without any specific information, default license of images is CC-BY-SA 3+.
|
|
|
|
|
|
## DOCUMENTATION
|
|
<pre>
|
|
cpm@ocmstar (13:19:32) ~/Projets/Logar/Dev/logar/dist/logar-0.1.1-SNAPSHOT_20210425131921 40 > ./logar.sh -h
|
|
2021-04-25T13:19:34.176928 Logar call: -h
|
|
Logar CLI version 0.1.1-20210425131921
|
|
Usage:
|
|
logar [ -h | -help | --help ]
|
|
logar [ -v | -version | --version ]
|
|
logar anonymize fileordirectory [mapfile] anonymize ip and user
|
|
logar archive [-dry] source target archive previous month from /var/log/nginx/ tree
|
|
logar check fileordirectory check line format in log file
|
|
logar checksort fileordirectory check sort in log file
|
|
logar sort fileordirectory sort log files by datetime
|
|
logar testconcate fileordirectory test line concate in log file
|
|
|
|
Done.
|
|
</pre>
|
|
|
|
### Command anonymize
|
|
|
|
This command is useful to anonymize log. It can anonymized accces log file and Nginx error log file from default Nginx combined format. The result is a new file whose name contains "-anon".
|
|
|
|
Optionally a map file can be set as parameter. Then the file is load and used to run. At the end, the file will be erased by a new one.
|
|
|
|
The anonymization does not change the line length:
|
|
- ip address: replaced with a valid random generate ip with the same length
|
|
- remote user: replaced with a random [a..z] string with the same length
|
|
|
|
Examples:
|
|
- logar anonymize test.log
|
|
- logar anonymize test.log.gz table
|
|
|
|
### Command archive
|
|
|
|
This command archive previous month log lines from a /var/log/nginx/ directory to single file by domain.
|
|
|
|
The optionally *-dry* parameter is useful to test the archive process without produced any target file.
|
|
|
|
Example:
|
|
- logar /var/log/nginx/ target/
|
|
- logar -dry /var/log/nginx/ target/
|
|
|
|
### Command check
|
|
|
|
This command check bad format line in log files (access or error). It is useful to detect malformed log file or test the logar parsing feature.
|
|
|
|
Example:
|
|
- logar check /var/log/nginx/date.chapril.org/
|
|
|
|
### Command checksort
|
|
|
|
This command check if log lines are sorted by datetime. It is useful to detect unsorted log file. Reminder: sorted file are mandatory to Webalizer.
|
|
|
|
Example:
|
|
- logar checksort /var/log/nginx/date.chapril.org/
|
|
- logar checksort /var/log/nginx/date.chapril.org/date.chapril.org-access-2021-01.log.gz
|
|
|
|
### Command sort
|
|
|
|
This command sorts log files. A backup file is produced. A check is made between original file and produced file.
|
|
|
|
Caution: as the log file is fully load in memory, huge file are potentially difficult.
|
|
|
|
Example:
|
|
- logar checksort /var/log/nginx/date.chapril.org/
|
|
- logar checksort /var/log/nginx/date.chapril.org/date.chapril.org-access-2021-01.log.gz
|
|
|
|
### Command testconcate
|
|
|
|
This command tests a advanced feature of rebuild log line. Perhaps useful feature in future.
|
|
|
|
## Requirements
|
|
|
|
- Java 11
|
|
- Eclipse 4.16 (202006).
|
|
|
|
## INSTALL
|
|
|
|
TODO
|
|
|
|
### Unit test environment
|
|
For unit tests, install the TestNG:
|
|
* https://marketplace.eclipse.org/content/testng-eclipse
|
|
* Eclipse menu > Help > Eclipse Marketplace > Find "TestNG" > TestNG for Eclipse: Install button
|