honey-be/LOGGING_GUIDE.md

# Logging Configuration Guide

## Overview

The application uses Logback for logging with the following features:
- **Runtime log level changes** (scan every 30 seconds)
- **Asynchronous file logging** (non-blocking I/O for high concurrency)
- **Automatic log rotation** (50MB per file, 14 days retention, 10GB total cap)
- **External configuration file** (editable on VPS without rebuilding)

## Log File Location

### Default Location
Logs are stored in: `./logs/` directory (relative to where the JAR is executed)

### Custom Location
Set the `LOG_DIR` environment variable or system property:
```bash
export LOG_DIR=/var/log/lottery-be
java -jar lottery-be.jar
```

Or:
```bash
java -DLOG_DIR=/var/log/lottery-be -jar lottery-be.jar
```

## Log File Naming

- **Current log**: `logs/lottery-be.log`
- **Rolled logs**: `logs/lottery-be-2024-01-15.0.log`, `logs/lottery-be-2024-01-15.1.log`, etc.
- **Max file size**: 50MB per file
- **Retention**: 14 days
- **Total size cap**: 10GB

## Using External logback-spring.xml on VPS

By default, `logback-spring.xml` is packaged inside the JAR. To use an external file on your VPS:

### Step 1: Copy logback-spring.xml to VPS

```bash
# Copy from JAR (if needed) or from your source code
# Place it in a location like: /opt/lottery-be/config/logback-spring.xml
# Or next to your JAR: /opt/lottery-be/logback-spring.xml
```

### Step 2: Start application with external config

```bash
# Option 1: System property
java -Dlogging.config=/opt/lottery-be/logback-spring.xml -jar lottery-be.jar

# Option 2: Environment variable
export LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml
java -jar lottery-be.jar

# Option 3: In systemd service file
[Service]
Environment="LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml"
ExecStart=/usr/bin/java -jar /opt/lottery-be/lottery-be.jar
```

### Step 3: Edit logback-spring.xml on VPS

```bash
# Edit the file
nano /opt/lottery-be/logback-spring.xml

# Change log level (example: change com.lottery from INFO to DEBUG)
# Find: <logger name="com.lottery" level="INFO"/>
# Change to: <logger name="com.lottery" level="DEBUG"/>

# Save and exit
# Logback will automatically reload within 30 seconds (scanPeriod="30 seconds")
```

## Linux Commands for Log Management

### Find Log Files

```bash
# Find all log files
find /opt/lottery-be -name "*.log" -type f

# Find logs in default location
ls -lh ./logs/

# Find logs with custom LOG_DIR
ls -lh /var/log/lottery-be/
```

### View Log Files

```bash
# View current log file (real-time)
tail -f logs/lottery-be.log

# View last 100 lines
tail -n 100 logs/lottery-be.log

# View with line numbers
cat -n logs/lottery-be.log | less

# View specific date's log
cat logs/lottery-be-2024-01-15.0.log
```

### Search Logs

```bash
# Search for errors
grep -i "error" logs/lottery-be.log

# Search for specific user ID
grep "userId=123" logs/lottery-be.log

# Search across all log files
grep -r "ERROR" logs/

# Search with context (5 lines before/after)
grep -C 5 "ERROR" logs/lottery-be.log

# Search and highlight
grep --color=always "ERROR\|WARN" logs/lottery-be.log | less -R
```

### Monitor Logs in Real-Time

```bash
# Follow current log
tail -f logs/lottery-be.log

# Follow and filter for errors only
tail -f logs/lottery-be.log | grep -i error

# Follow multiple log files
tail -f logs/lottery-be*.log

# Follow with timestamps
tail -f logs/lottery-be.log | while read line; do echo "[$(date '+%Y-%m-%d %H:%M:%S')] $line"; done
```

### Check Log File Sizes

```bash
# Check size of all log files
du -sh logs/*

# Check total size of logs directory
du -sh logs/

# List files sorted by size
ls -lhS logs/

# Check disk space
df -h
```

### Clean Old Logs

```bash
# Logback automatically deletes logs older than 14 days
# But you can manually clean if needed:

# Remove logs older than 7 days
find logs/ -name "*.log" -mtime +7 -delete

# Remove logs older than 14 days (matching logback retention)
find logs/ -name "*.log" -mtime +14 -delete
```

## Changing Log Level at Runtime

### Method 1: Edit logback-spring.xml (Recommended)

1. **Edit the external logback-spring.xml file**:
   ```bash
   nano /opt/lottery-be/logback-spring.xml
   ```

2. **Change the logger level** (example: enable DEBUG for entire app):
   ```xml
   <!-- Change from: -->
   <logger name="com.lottery" level="INFO"/>
   
   <!-- To: -->
   <logger name="com.lottery" level="DEBUG"/>
   ```

3. **Save the file**. Logback will automatically reload within 30 seconds.

4. **Verify the change**:
   ```bash
   tail -f logs/lottery-be.log
   # You should see DEBUG logs appearing after ~30 seconds
   ```

### Method 2: Change Specific Logger

To change only a specific service (e.g., GameRoomService):

```xml
<!-- In logback-spring.xml, change: -->
<logger name="com.lottery.lottery.service.GameRoomService" level="WARN"/>

<!-- To: -->
<logger name="com.lottery.lottery.service.GameRoomService" level="DEBUG"/>
```

### Method 3: Change Root Level

To change the root level for all loggers:

```xml
<!-- In logback-spring.xml, change: -->
<root level="INFO">

<!-- To: -->
<root level="DEBUG">
```

**Note**: This will generate A LOT of logs. Use with caution in production.

## Log Levels Explained

- **ERROR**: Critical errors that need immediate attention
- **WARN**: Warnings that might indicate problems
- **INFO**: Important application events (round completion, payments, etc.)
- **DEBUG**: Detailed debugging information (very verbose, use only for troubleshooting)

## Default Configuration

- **Root level**: INFO
- **Application (com.lottery)**: INFO
- **High-traffic services**: WARN (GameRoomService, GameWebSocketController)
- **Infrastructure packages**: WARN (Spring, Hibernate, WebSocket, etc.)

## Performance Considerations

- **Asynchronous logging**: Logs are written asynchronously to prevent blocking main threads
- **Queue size**: 256 log entries (good for 1000+ concurrent users)
- **Never block**: If queue is full, lower-level logs (DEBUG/INFO) may be dropped, but WARN/ERROR are always kept
- **File I/O**: All file writes are non-blocking

## Troubleshooting

### Logs not appearing

1. Check log file location:
   ```bash
   ls -la logs/
   ```

2. Check file permissions:
   ```bash
   ls -l logs/lottery-be.log
   # Ensure the application user has write permissions
   ```

3. Check disk space:
   ```bash
   df -h
   ```

### Log level changes not taking effect

1. Verify scan is enabled in logback-spring.xml:
   ```xml
   <configuration scan="true" scanPeriod="30 seconds">
   ```

2. Check for syntax errors in logback-spring.xml:
   ```bash
   # Logback will log errors to console if config is invalid
   ```

3. Restart application if needed (shouldn't be necessary with scan enabled)

### Too many logs / Out of memory

1. Increase log level to WARN:
   ```xml
   <root level="WARN">
   ```

2. Check log file sizes:
   ```bash
   du -sh logs/*
   ```

3. Clean old logs manually if needed

## Example: Enabling DEBUG for Troubleshooting

1. **Edit logback-spring.xml**:
   ```bash
   nano /opt/lottery-be/logback-spring.xml
   ```

2. **Change specific logger to DEBUG**:
   ```xml
   <logger name="com.lottery.lottery.service.GameRoomService" level="DEBUG"/>
   ```

3. **Save and wait 30 seconds**

4. **Monitor logs**:
   ```bash
   tail -f logs/lottery-be.log | grep "GameRoomService"
   ```

5. **After troubleshooting, change back to WARN**:
   ```xml
   <logger name="com.lottery.lottery.service.GameRoomService" level="WARN"/>
   ```

## Systemd Service Example

If using systemd, here's an example service file:

```ini
[Unit]
Description=Lottery Backend Application
After=network.target

[Service]
Type=simple
User=lottery
WorkingDirectory=/opt/lottery-be
Environment="LOGGING_CONFIG=/opt/lottery-be/logback-spring.xml"
Environment="LOG_DIR=/var/log/lottery-be"
ExecStart=/usr/bin/java -jar /opt/lottery-be/lottery-be.jar
Restart=always
RestartSec=10

[Install]
WantedBy=multi-user.target
```