Files
honey-be/LOGGING_GUIDE.md
Tihon 15498c8337
All checks were successful
Deploy to VPS / deploy (push) Successful in 52s
Initial setup, cleanup, VPS setup
2026-03-07 23:11:31 +02:00

7.8 KiB

Logging Configuration Guide

Overview

The application uses Logback for logging with the following features:

  • Runtime log level changes (scan every 30 seconds)
  • Asynchronous file logging (non-blocking I/O for high concurrency)
  • Automatic log rotation (50MB per file, 14 days retention, 10GB total cap)
  • External configuration file (editable on VPS without rebuilding)

Log File Location

Default Location

Logs are stored in: ./logs/ directory (relative to where the JAR is executed)

Custom Location

Set the LOG_DIR environment variable or system property:

export LOG_DIR=/var/log/lottery-be
java -jar lottery-be.jar

Or:

java -DLOG_DIR=/var/log/lottery-be -jar lottery-be.jar

Log File Naming

  • Current log: logs/lottery-be.log
  • Rolled logs: logs/lottery-be-2024-01-15.0.log, logs/lottery-be-2024-01-15.1.log, etc.
  • Max file size: 50MB per file
  • Retention: 14 days
  • Total size cap: 10GB

Using External logback-spring.xml on VPS

By default, logback-spring.xml is packaged inside the JAR. To use an external file on your VPS:

Step 1: Copy logback-spring.xml to VPS

# Copy from JAR (if needed) or from your source code
# Place it in a location like: /opt/lottery-be/config/logback-spring.xml
# Or next to your JAR: /opt/lottery-be/logback-spring.xml

Step 2: Start application with external config

# Option 1: System property
java -Dlogging.config=/opt/lottery-be/logback-spring.xml -jar lottery-be.jar

# Option 2: Environment variable
export LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml
java -jar lottery-be.jar

# Option 3: In systemd service file
[Service]
Environment="LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml"
ExecStart=/usr/bin/java -jar /opt/lottery-be/lottery-be.jar

Step 3: Edit logback-spring.xml on VPS

# Edit the file
nano /opt/lottery-be/logback-spring.xml

# Change log level (example: change com.lottery from INFO to DEBUG)
# Find: <logger name="com.lottery" level="INFO"/>
# Change to: <logger name="com.lottery" level="DEBUG"/>

# Save and exit
# Logback will automatically reload within 30 seconds (scanPeriod="30 seconds")

Linux Commands for Log Management

Find Log Files

# Find all log files
find /opt/lottery-be -name "*.log" -type f

# Find logs in default location
ls -lh ./logs/

# Find logs with custom LOG_DIR
ls -lh /var/log/lottery-be/

View Log Files

# View current log file (real-time)
tail -f logs/lottery-be.log

# View last 100 lines
tail -n 100 logs/lottery-be.log

# View with line numbers
cat -n logs/lottery-be.log | less

# View specific date's log
cat logs/lottery-be-2024-01-15.0.log

Search Logs

# Search for errors
grep -i "error" logs/lottery-be.log

# Search for specific user ID
grep "userId=123" logs/lottery-be.log

# Search across all log files
grep -r "ERROR" logs/

# Search with context (5 lines before/after)
grep -C 5 "ERROR" logs/lottery-be.log

# Search and highlight
grep --color=always "ERROR\|WARN" logs/lottery-be.log | less -R

Monitor Logs in Real-Time

# Follow current log
tail -f logs/lottery-be.log

# Follow and filter for errors only
tail -f logs/lottery-be.log | grep -i error

# Follow multiple log files
tail -f logs/lottery-be*.log

# Follow with timestamps
tail -f logs/lottery-be.log | while read line; do echo "[$(date '+%Y-%m-%d %H:%M:%S')] $line"; done

Check Log File Sizes

# Check size of all log files
du -sh logs/*

# Check total size of logs directory
du -sh logs/

# List files sorted by size
ls -lhS logs/

# Check disk space
df -h

Clean Old Logs

# Logback automatically deletes logs older than 14 days
# But you can manually clean if needed:

# Remove logs older than 7 days
find logs/ -name "*.log" -mtime +7 -delete

# Remove logs older than 14 days (matching logback retention)
find logs/ -name "*.log" -mtime +14 -delete

Changing Log Level at Runtime

  1. Edit the external logback-spring.xml file:

    nano /opt/lottery-be/logback-spring.xml
    
  2. Change the logger level (example: enable DEBUG for entire app):

    <!-- Change from: -->
    <logger name="com.lottery" level="INFO"/>
    
    <!-- To: -->
    <logger name="com.lottery" level="DEBUG"/>
    
  3. Save the file. Logback will automatically reload within 30 seconds.

  4. Verify the change:

    tail -f logs/lottery-be.log
    # You should see DEBUG logs appearing after ~30 seconds
    

Method 2: Change Specific Logger

To change only a specific service (e.g., GameRoomService):

<!-- In logback-spring.xml, change: -->
<logger name="com.lottery.lottery.service.GameRoomService" level="WARN"/>

<!-- To: -->
<logger name="com.lottery.lottery.service.GameRoomService" level="DEBUG"/>

Method 3: Change Root Level

To change the root level for all loggers:

<!-- In logback-spring.xml, change: -->
<root level="INFO">

<!-- To: -->
<root level="DEBUG">

Note: This will generate A LOT of logs. Use with caution in production.

Log Levels Explained

  • ERROR: Critical errors that need immediate attention
  • WARN: Warnings that might indicate problems
  • INFO: Important application events (round completion, payments, etc.)
  • DEBUG: Detailed debugging information (very verbose, use only for troubleshooting)

Default Configuration

  • Root level: INFO
  • Application (com.lottery): INFO
  • High-traffic services: WARN (GameRoomService, GameWebSocketController)
  • Infrastructure packages: WARN (Spring, Hibernate, WebSocket, etc.)

Performance Considerations

  • Asynchronous logging: Logs are written asynchronously to prevent blocking main threads
  • Queue size: 256 log entries (good for 1000+ concurrent users)
  • Never block: If queue is full, lower-level logs (DEBUG/INFO) may be dropped, but WARN/ERROR are always kept
  • File I/O: All file writes are non-blocking

Troubleshooting

Logs not appearing

  1. Check log file location:

    ls -la logs/
    
  2. Check file permissions:

    ls -l logs/lottery-be.log
    # Ensure the application user has write permissions
    
  3. Check disk space:

    df -h
    

Log level changes not taking effect

  1. Verify scan is enabled in logback-spring.xml:

    <configuration scan="true" scanPeriod="30 seconds">
    
  2. Check for syntax errors in logback-spring.xml:

    # Logback will log errors to console if config is invalid
    
  3. Restart application if needed (shouldn't be necessary with scan enabled)

Too many logs / Out of memory

  1. Increase log level to WARN:

    <root level="WARN">
    
  2. Check log file sizes:

    du -sh logs/*
    
  3. Clean old logs manually if needed

Example: Enabling DEBUG for Troubleshooting

  1. Edit logback-spring.xml:

    nano /opt/lottery-be/logback-spring.xml
    
  2. Change specific logger to DEBUG:

    <logger name="com.lottery.lottery.service.GameRoomService" level="DEBUG"/>
    
  3. Save and wait 30 seconds

  4. Monitor logs:

    tail -f logs/lottery-be.log | grep "GameRoomService"
    
  5. After troubleshooting, change back to WARN:

    <logger name="com.lottery.lottery.service.GameRoomService" level="WARN"/>
    

Systemd Service Example

If using systemd, here's an example service file:

[Unit]
Description=Lottery Backend Application
After=network.target

[Service]
Type=simple
User=lottery
WorkingDirectory=/opt/lottery-be
Environment="LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml"
Environment="LOG_DIR=/var/log/lottery-be"
ExecStart=/usr/bin/java -jar /opt/lottery-be/lottery-be.jar
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target