Turn a robot simulator trace into a trustworthy command-line report. Build a small C program that reads bounded lines, validates records, accumulates statistics, and explains failures.
Get a tiny vertical slice working: one valid row enters stdin and produces one correct report. Learn the data shape before the error branches.
Extend the one-row path with a strict parser contract: only a whole valid row can create a record.
Extend one-row input to a sequence without keeping the whole file in memory or mistaking a damaged tail for another row.
Turn many validated records into useful counts and latency aggregates using fixed-size state.
Deliver a real CLI with a documented exit policy, clear cleanup, reproducible fixtures, and tests that can catch incorrect results.
Leon is testing a classroom robot simulator. Each simulated action writes a severity and a duration, such as WARN 40. A slow action is easy to miss in a long trace, and a damaged row can make a careless average misleading. Leon starts with one good row, defines exactly what a record means, then adds a stream reader and a running summary. The finished tool reports how many INFO, WARN, and ERROR events occurred, their latency range and mean, and how many rows were rejected. All data is synthetic. This is an original engineering project and its own concept checks, not a supplied course exercise or solution.
One clear interface makes the first CLI easy to use and test; there is no hidden interactive menu.
Check: No argument or two arguments gives exit 2 and no stdout. A copied demo file and piped stdin both work.
A precise grammar prevents 12ms, -1, 1.5, or extra columns from silently becoming valid measurements.
Check: Test both endpoints and one value above the maximum, plus unknown levels, signs, suffixes, and an extremely long integer.
The fixed buffer needs one extra byte for its terminator. Draining a bad line stops its remaining bytes from pretending to be another row.
Check: A 127-byte row succeeds; a 128-byte row fails; a valid row immediately after it still counts once.
This bounds byte-processing work and gives a simple proof that counters and duration totals cannot overflow.
Check: Exactly 1 MiB of valid padded rows succeeds. One additional byte yields exit 1 and empty stdout.
The person reading the report can judge missing data, while scripts can detect a non-clean run.
Check: The mixed fixture reports 3 valid and 3 malformed rows; a nonexistent file returns 1 with no report.
No observation is not the same as a zero-duration observation, and dividing by zero would be invalid.
Check: Hand-check the demo sum 200 and mean 33.333; check all n/a values on the empty fixture.
Validate argc, open a file or borrow stdin, connect reader → parser → statistics, close an owned file, and choose the exit status. This is where policy lives.
Read a byte stream into the caller’s fixed buffer. Distinguish complete line, malformed framing, clean EOF, I/O failure, and input limit. Drain bad lines within the total byte budget.
Read a complete C string without changing it. Validate level, separators, digits, numeric bound, and end of text. Commit the local candidate only after every check passes.
Maintain line/count/sum/extrema invariants without storing records. Format one final report and flush it so buffered output errors can be noticed.
argv → FILE stream → bounded physical line → validated record → running statistics → stdout. Bad records take a side path to malformed count and bounded stderr warnings. Fatal input errors stop before stdout. No application thread or heap-owned record is needed.
| Resource | Owner | Release |
|---|---|---|
| FILE * returned by fopen | main opens the file and lends the pointer to analyze and read_line. | main calls fclose exactly once after analysis, including after an input failure. |
| stdin and stdout | The C runtime provides these streams. This project borrows them. | Do not fclose borrowed stdin. stats_print flushes stdout; normal process shutdown performs runtime cleanup. |
| 128-byte line buffer and parsed record | analyze owns local stack storage. read_line writes the buffer; parse_record copies values into the record. | The buffer is reused only after the current row is consumed. No function retains its pointer; local storage ends when its scope ends. |
| Statistics structure | main owns a zero-initialized struct and lends it for updates and printing. | No free is needed: it has automatic storage duration and contains values, not allocated pointers. |
Get a tiny vertical slice working: one valid row enters stdin and produces one correct report. Learn the data shape before the error branches.
Start with a question small enough to answer by hand: what should INFO 12 produce? It describes one event, so valid and INFO must be 1; total, min, max, and mean must all describe 12 ms. This expectation connects an input rule to an observable result. Use the complete downloaded files as a working baseline while understanding one path at a time. A struct groups related values so you can pass one record without confusing its level with its duration.
In the project directory run make, then printf 'INFO 12\n' | ./log_analyzer -. Keep the terminal output next to your hand prediction. The dash is an argument; the shell pipe connects printf output to the program’s stdin.
Read the record.h extract. The enum gives readable names to three categories; LEVEL_COUNT sizes a three-element counter array later. unsigned stores a nonnegative duration, but the parser must still enforce the smaller 60000 limit.
Trace only the successful calls in main.c: read_line fills text, parse_record fills a local record, stats_accept updates totals, and stats_print creates the report. Circle where text becomes typed values. Do not copy extracts into a new standalone file; build all supplied source files together.
Teaching extract from the complete project. Build the full files below to run it.
enum log_level { LEVEL_INFO, LEVEL_WARN, LEVEL_ERROR, LEVEL_COUNT };
struct record {
enum log_level level;
unsigned duration_ms;
};
Extract from record.h. The record contains values rather than a pointer into the input line. A later read can overwrite that line without changing a record already passed to the statistics code. This is the first ownership decision, even though no malloc appears.
No. ERROR is a valid severity in this format. It changes the ERROR counter to 1, and the process still returns 0 because the input itself is valid.
Extend the one-row path with a strict parser contract: only a whole valid row can create a record.
A conversion routine that reads the prefix 12 from 12ms would hide an input mistake. Our parser consumes a complete level token, requires a separator, reads at least one digit, and checks that only allowed trailing whitespace remains. It builds a local candidate and assigns *out once at the end. The caller therefore never receives half of a new record after failure. For each digit d, the next value would be 10v+d. Rewriting 10v+d <= 60000 as v <= (60000-d)/10 allows checking before multiplication.
Read the promise above parse_record in record.h, then its implementation. Mark input as borrowed and const; mark out as caller-owned writable storage. Neither pointer may be kept after the call.
Follow candidate through INFO 60000, digit by digit. Before the final zero, v is 6000 and the check permits it. For INFO 60001, the final digit makes the threshold 5999, so parsing stops before writing an out-of-range value.
Run make test and locate the numeric-grammar test. Predict rejection for +1, -1, 1.5, 12ms, 1 2, and 60001 before reading assertions. Test one malformed row after a good row and verify only the good duration enters the sum.
Teaching extract from the complete project. Build the full files below to run it.
while (*p >= '0' && *p <= '9') {
unsigned digit = (unsigned)(*p - '0');
if (candidate.duration_ms > (MAX_DURATION_MS - digit) / 10U) {
return false;
}
candidate.duration_ms = candidate.duration_ms * 10U + digit;
++p;
}
if (*skip_space(p) != '\0') {
return false;
}
*out = candidate;
return true;
Extract from record.c after the level and first-digit checks. The bound check happens before multiply-and-add. The final NUL check rejects suffixes and extra fields. *out = candidate is the commit point: all previous branches return without touching caller output.
A later duration error would leave output partly changed. A local candidate makes the interface easier to reason about: success replaces the whole record; failure replaces nothing.
Extend one-row input to a sequence without keeping the whole file in memory or mistaking a damaged tail for another row.
The reader handles bytes and line endings; the parser handles the meaning of a complete line. Keeping those jobs separate makes failures easier to locate. A 127-byte payload needs an array of 128 bytes because C strings also need a NUL terminator. Once a row is known to be bad, stopping immediately would leave its tail waiting in the stream. Keep reading until LF or EOF, but keep enforcing the total byte budget. EOF alone is not proof of success: ferror distinguishes a read failure.
Read the five line_status meanings in reader.h before the loop. main may parse only LINE_OK. LINE_BAD increments rejection once; LINE_END finishes; LINE_IO_ERROR and LINE_LIMIT abort without a report.
Trace used and bad in the extract for 127 bytes, then a 128th byte. used never exceeds 127. After bad becomes true, writing stops but reading continues. An embedded NUL follows the same rejection path, preventing hidden text after a fake string terminator.
Use the line-limit and binary-byte tests to check recovery. Then inspect the EOF branch for a last row without LF and the CR removal for CRLF. The 1 MiB test checks a different boundary: total input, not per-line capacity.
Teaching extract from the complete project. Build the full files below to run it.
if (ch == '\n') {
break;
}
if (ch == '\0' || used == MAX_LINE_BYTES) {
bad = true;
}
if (!bad) {
line[used++] = (char)ch;
}
/* Keep consuming a bad line so its tail is not a new record. */
Extract from reader.c, inside its byte loop. fgetc is stored in int elsewhere in the same loop, so EOF stays distinct from byte values. The buffer belongs to analyze; read_line borrows it. Do not read its contents after LINE_BAD, because no complete C string is promised on that path.
The input contract counts physical lines, not buffer-sized chunks. Draining preserves that framing, so the next call starts at the following real line.
Turn many validated records into useful counts and latency aggregates using fixed-size state.
You do not need to remember every duration to compute a count, sum, minimum, or maximum. After each complete row, lines = valid + malformed. After each valid row, INFO + WARN + ERROR = valid. Only valid records affect latency metrics. Use the first valid duration to initialize both extrema; otherwise a zero-initialized minimum would stay zero for all-positive data. uint64_t holds counts and sums because their conservative bounds exceed small integer ranges, while a single 0–60000 duration fits unsigned.
Initialize totals with {0}, then read stats_accept and stats_reject. Trace the first two demo rows: after INFO 10 and WARN 40, count is 2, total is 50, min is 10 and max is 40. Confirm no rejected duration reaches this function.
Finish the six-row hand calculation before running the demo. The sum is 200 and 200/6 is 33.333… . In stats_print, cast before division and display three decimal places. Integer aggregation stays exact even though mean display is rounded.
Run the empty fixture. valid is zero, so no observed min or max exists and division is forbidden. Find the n/a branch, then run the positive-only mixed fixture to catch the minimum-initialization bug that the zero-containing demo alone would hide.
Teaching extract from the complete project. Build the full files below to run it.
void stats_accept(struct stats *s, const struct record *record)
{
if (s->valid == 0 || record->duration_ms < s->min_ms) {
s->min_ms = record->duration_ms;
}
if (s->valid == 0 || record->duration_ms > s->max_ms) {
s->max_ms = record->duration_ms;
}
++s->lines;
++s->valid;
++s->by_level[record->level];
s->total_ms += record->duration_ms;
}
Extract from stats.c. Update extrema before increasing valid so valid == 0 means this is the first accepted row. record->level is safe as an index because the caller must pass a record produced by successful parsing. The header states that precondition; this is an internal interface, not an arbitrary-data API.
No. Count, sum, min and max lose the ordering information needed for a median. That extension needs another representation, such as a bounded histogram, and a new memory/correctness argument.
Deliver a real CLI with a documented exit policy, clear cleanup, reproducible fixtures, and tests that can catch incorrect results.
A plausible report is not enough. The caller also needs to know whether every byte was inspected and whether any records were excluded. main owns the decision: 0 for clean completion, 3 for completed input with malformed rows, 2 for usage, and 1 for input/capacity or reported output failures. Close an owned file even when analyze fails. Delay stdout until input and close succeed; an output error may still leave a prefix. Tests compare real subprocess behavior with independent expected answers, and place temporary inputs in their own directories.
Read main from argument validation to return. Mark owns_input when fopen supplies the stream and false when stdin is borrowed. Follow the excerpt’s cleanup on success and failure; there is one fclose location and no free for stack-owned totals.
Run the mixed fixture and check echo $? immediately: it is 3 despite useful stdout. Run a missing filename and no argument: expect 1 and 2 with empty stdout. stderr is a separate diagnostic channel, so a script can keep the report apart from warnings.
Run make test. For each test, name the bug it would catch: wrong mean type, permissive prefix parsing, incomplete draining, lost exit code, or a byte-budget off-by-one. The tests use a five-second subprocess timeout and literal expected totals. They do not simulate every output/close failure or impose a runtime deadline on ordinary CLI use.
Teaching extract from the complete project. Build the full files below to run it.
struct stats totals = {0};
bool completed = analyze(input, &totals);
if (owns_input && fclose(input) != 0) {
fputs("error: cannot close input\n", stderr);
completed = false;
}
if (!completed) {
return EXIT_IO;
}
if (!stats_print(stdout, &totals)) {
fputs("error: cannot write report\n", stderr);
return EXIT_IO;
}
return totals.malformed == 0 ? EXIT_CLEAN : EXIT_DATA;
Extract from the end of main.c. completed records whether input processing really finished. fclose runs for owned input before this result is checked. stats_print checks buffered output with fflush; its failure also becomes exit 1. A malformed completed input is distinguished at the final return.
The report can be correct for accepted rows while silently hiding data loss. Exit 3 and line warnings tell a caller about rejected rows. Testing all three channels checks the tool’s full public contract.
Use Clang with C11 support, make and Python 3 on macOS or Linux. On Windows use a Linux environment such as WSL. The supplied Makefiles use Clang. Open a terminal in the download directory. Unzip the archive, then enter its folder. make builds the executable; make test checks its behavior.
unzip comp2017-log-analyzer.zip
cd comp2017-log-analyzer
make
make test
./log_analyzer fixtures/demo.log
lines=6 valid=6 malformed=0 INFO=3 WARN=2 ERROR=1 total_ms=200 min_ms=0 max_ms=100 mean_ms=33.333
Exit status: 0.
After make, run this from the project directory. Exit 0; stderr is empty. Six accepted rows sum to 200 ms, so 200/6 displays as 33.333. ERROR=1 is an event count, not a failed process.
./log_analyzer fixtures/mixed.log
lines=6 valid=3 malformed=3 INFO=1 WARN=1 ERROR=1 total_ms=30 min_ms=5 max_ms=15 mean_ms=10.000
Exit status: 3.
Exit 3. stdout summarizes only the three valid rows. stderr contains warning: line 2: malformed record, warning: line 4: malformed record, and warning: line 5: malformed record, each on its own line. DEBUG, -1, and 60001 are rejected. Run echo $? immediately afterward to inspect the exit status.
./log_analyzer fixtures/empty.log
lines=0 valid=0 malformed=0 INFO=0 WARN=0 ERROR=0 total_ms=0 min_ms=n/a max_ms=n/a mean_ms=n/a
Exit status: 0.
Exit 0; stderr is empty. The zero-byte fixture contains no physical lines. n/a explicitly avoids inventing an observed minimum, maximum, or average when the valid count is zero.
Zero initialization was mistaken for a real observation.
Run two rows with durations 5 and 15. Inspect s->valid just before the first stats_accept update.
When valid is zero, set both extrema from the first real record before incrementing valid. Keep the positive-only mixed fixture test.
Integer division discarded the fraction before conversion to double.
Write down 5/3 = 1.666… and inspect the operand types in stats_print, not just the destination type.
Convert before division: (double)total_ms / (double)valid. The fractional-mean test expects 1.667.
The reader returned when its buffer filled, leaving part of the same physical line unread.
Place a 128-byte row before WARN 7. The final report must have two physical lines and exactly one malformed row.
After marking a row bad, keep consuming through LF or EOF, still enforcing the whole-input budget. Never parse a LINE_BAD buffer.
The parser validated only a numeric prefix and ignored the remaining text.
After the digit loop, inspect the character after skipped trailing spaces. It must be the terminating NUL.
Reject any remaining non-whitespace text and assign *out only at the end. Test suffixes and extra fields as distinct failures.
EOF was treated as proof of a clean end without inspecting ferror.
Use the directory-input test on macOS/Linux; open may fail immediately or reading the opened directory may fail. In either case stdout must be empty.
Check ferror when fgetc returns EOF. Propagate LINE_IO_ERROR through analyze so main returns 1 without stats_print.
Exit 0; 3 INFO, 2 WARN, 1 ERROR; sum 200; range 0–100; mean 33.333.
Catches wrong severity indexing, missing updates, and wrong aggregates using values computed independently of the program.
Exit 3; 3 valid and 3 malformed; total 30; warnings identify lines 2, 4, 5.
Checks that rejection changes only rejection counters and that later good records still work.
Empty input: exit 0, zero counts, n/a latencies. Blank rows: exit 3, twenty rejections, five detailed warnings and one omission message.
Separates absence of data from bad data and prevents unbounded diagnostic output.
Accept 0, 60000 and leading zeroes; reject 60001, signs, fractions, suffixes, huge integers, extra fields, and nonmatching levels.
Exercises both edges of the contract and stops permissive partial conversion.
Accept CRLF, tabs and a final row without LF. Reject NUL and 0xff rows without losing the following ERROR 9.
Checks string termination, the distinction between bytes and EOF, and continued processing after damage.
Exactly 3 lines, 2 accepted and 1 rejected; total duration 8.
Finds off-by-one buffer errors and verifies drain-to-newline behavior.
8192 padded valid rows succeed at exactly 1,048,576 bytes. Adding one byte returns 1 with no report.
The accepted side and rejected side jointly verify the whole-input work bound.
Wrong argument count gives 2; missing file or directory input gives 1; every case has empty stdout.
Exercises actual subprocesses and filesystem behavior, without mocking successful output.
Exit 0 and mean_ms=1.667.
Detects truncating integer division that whole-number averages would miss.
A growing getline buffer is convenient, but a fixed 128-byte array makes memory and termination visible in a first project. We reject larger rows and drain them. Time is O(B) for bytes consumed, application state is O(1), and at most one byte beyond 1 MiB is inspected. Standard I/O owns additional internal buffers. A byte budget is not a time deadline: stdin can block waiting for its producer, so start with completed regular files.
A duration is unsigned and at most 60000. The parser checks before multiplying. Counts and sums use uint64_t; even the conservative bound 1,048,576 × 60,000 = 62,914,560,000 fits. A static assertion protects this bound when constants change. The mean uses floating division only after integer aggregation; display rounding to three decimals does not change the exact integer sum.
Malformed rows do not destroy valid observations. Exit 3 communicates that the completed report excludes some rows. Fatal input failures instead suppress stdout because the input was not fully inspected. Warnings are capped at five detailed lines plus one omission message. This makes the tool useful in a terminal and predictable in scripts. A valid ERROR event alone never makes the process fail.
Changing whitespace rules belongs in record.c; changing line capacity belongs in reader.c; adding an aggregate belongs in stats.c; changing CLI policy belongs in main.c. Headers document borrowed pointers and preconditions. The parser promises not to retain text pointers, so no record depends on the reused buffer. The source remains four implementation files rather than one file for every helper.
The format is synthetic: no timestamps, arbitrary messages, live tailing, or percentiles. Returned read/write/close errors are checked, but a write error can leave part of stdout and a broken pipe can trigger the normal POSIX SIGPIPE signal. stderr failures are not retried. The tests cover deterministic contract behavior, not every device or filesystem failure. No timing or scheduler outcome is used as a correct answer.
These are the complete files in the download. The numbers suggest a reading order. The file list links directly to each source block.
Record interface
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#ifndef RECORD_H
#define RECORD_H
#include <stdbool.h>
#define MAX_DURATION_MS 60000U
enum log_level { LEVEL_INFO, LEVEL_WARN, LEVEL_ERROR, LEVEL_COUNT };
struct record {
enum log_level level;
unsigned duration_ms;
};
/* line: readable NUL-terminated text; out: writable caller-owned record.
* Return true and assign *out only for a complete valid record.
* Neither pointer is retained; a failed parse leaves *out unchanged. */
bool parse_record(const char *line, struct record *out);
#endif
Read the enum and struct together: severity is a category, duration is a bounded nonnegative value.
The parser receives borrowed text and a caller-owned output address; success is reported as bool.
The failure promise is explicit: *out stays unchanged. Check the implementation against that promise.
Strict complete-row parsing
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#include "record.h"
#include <stddef.h>
#include <string.h>
static bool is_space(char ch)
{
return ch == ' ' || ch == '\t';
}
static const char *skip_space(const char *p)
{
while (is_space(*p)) {
++p;
}
return p;
}
bool parse_record(const char *line, struct record *out)
{
struct record candidate = {LEVEL_INFO, 0U};
const char *start = skip_space(line);
const char *p = start;
while (*p != '\0' && !is_space(*p)) {
++p;
}
size_t length = (size_t)(p - start);
if (length == 4 && memcmp(start, "INFO", 4) == 0) {
candidate.level = LEVEL_INFO;
} else if (length == 4 && memcmp(start, "WARN", 4) == 0) {
candidate.level = LEVEL_WARN;
} else if (length == 5 && memcmp(start, "ERROR", 5) == 0) {
candidate.level = LEVEL_ERROR;
} else {
return false;
}
if (!is_space(*p)) {
return false;
}
p = skip_space(p);
if (*p < '0' || *p > '9') {
return false;
}
while (*p >= '0' && *p <= '9') {
unsigned digit = (unsigned)(*p - '0');
if (candidate.duration_ms > (MAX_DURATION_MS - digit) / 10U) {
return false;
}
candidate.duration_ms = candidate.duration_ms * 10U + digit;
++p;
}
if (*skip_space(p) != '\0') {
return false;
}
*out = candidate;
return true;
}
The two static whitespace helpers accept only space and tab, keeping the grammar independent of locale.
Token length and memcmp verify a whole severity word before numeric parsing starts.
The digit loop checks range before arithmetic; the final end-of-string test and assignment implement all-or-nothing output.
Line and byte-budget interface
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#ifndef READER_H
#define READER_H
#include <stddef.h>
#include <stdio.h>
#define MAX_LINE_BYTES 127U
#define MAX_INPUT_BYTES 1048576U
enum line_status { LINE_OK, LINE_BAD, LINE_END, LINE_IO_ERROR, LINE_LIMIT };
struct line_reader {
FILE *stream; /* Borrowed: only the caller closes this stream. */
size_t bytes_read; /* Start at zero; includes newline and CR bytes. */
};
/* Caller supplies MAX_LINE_BYTES + 1 writable bytes.
* LINE_OK: complete NUL-terminated line, without LF or a final CR.
* LINE_BAD: oversized or NUL-containing line was consumed through LF/EOF.
* LINE_END: clean EOF with no pending line. Other results are fatal.
* LINE_LIMIT may consume one byte beyond the budget to distinguish EOF.
* Do not parse the buffer unless the result is LINE_OK. */
enum line_status read_line(struct line_reader *reader,
char line[MAX_LINE_BYTES + 1U]);
#endif
MAX_LINE_BYTES excludes LF but includes CR; add one byte when allocating a C string buffer.
A line_reader holds a borrowed stream and a cumulative byte count. Initialize that count once per input.
Each enum result has a different caller action; only LINE_OK promises parseable buffer contents.
Physical-line framing
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#include "reader.h"
#include <stdbool.h>
enum line_status read_line(struct line_reader *reader,
char line[MAX_LINE_BYTES + 1U])
{
size_t used = 0;
bool seen_byte = false;
bool bad = false;
for (;;) {
int ch = fgetc(reader->stream);
if (ch == EOF) {
if (ferror(reader->stream)) {
return LINE_IO_ERROR;
}
if (!seen_byte) {
return LINE_END;
}
break;
}
if (reader->bytes_read == MAX_INPUT_BYTES) {
return LINE_LIMIT;
}
++reader->bytes_read;
seen_byte = true;
if (ch == '\n') {
break;
}
if (ch == '\0' || used == MAX_LINE_BYTES) {
bad = true;
}
if (!bad) {
line[used++] = (char)ch;
}
/* Keep consuming a bad line so its tail is not a new record. */
}
if (bad) {
return LINE_BAD;
}
if (used > 0 && line[used - 1] == '\r') {
--used;
}
line[used] = '\0';
return LINE_OK;
}
fgetc returns int. On EOF check ferror before deciding whether an unfinished last line exists.
Every consumed byte contributes to the input budget, including newline and discarded bytes.
bad freezes buffer writes but not reading. A successful line is CR-normalized and NUL-terminated before return.
State and statistics interface
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#ifndef STATS_H
#define STATS_H
#include "record.h"
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
struct stats {
uint64_t lines;
uint64_t valid;
uint64_t malformed;
uint64_t by_level[LEVEL_COUNT];
uint64_t total_ms;
unsigned min_ms;
unsigned max_ms;
};
/* Start with struct stats s = {0}. Add at most MAX_INPUT_BYTES records.
* record must come from a successful parse_record call.
* All pointers are borrowed for the duration of each call. */
void stats_accept(struct stats *s, const struct record *record);
void stats_reject(struct stats *s);
/* Write the final summary; false means a write/flush error.
* A failed output may already contain a prefix. Do not close output here. */
bool stats_print(FILE *output, const struct stats *s);
#endif
Separate valid from malformed so the report cannot confuse total lines with accepted observations.
Use uint64_t for repeated counts and sums; extrema have the same type as one duration.
Read the preconditions: zero-initialized state, validated records, and no more than the input budget of updates.
Updates and report formatting
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#include "stats.h"
#include "reader.h"
#include <inttypes.h>
_Static_assert((uint64_t)MAX_INPUT_BYTES <= UINT64_MAX / MAX_DURATION_MS,
"The byte budget must keep duration totals representable");
void stats_accept(struct stats *s, const struct record *record)
{
if (s->valid == 0 || record->duration_ms < s->min_ms) {
s->min_ms = record->duration_ms;
}
if (s->valid == 0 || record->duration_ms > s->max_ms) {
s->max_ms = record->duration_ms;
}
++s->lines;
++s->valid;
++s->by_level[record->level];
s->total_ms += record->duration_ms;
}
void stats_reject(struct stats *s)
{
++s->lines;
++s->malformed;
}
bool stats_print(FILE *output, const struct stats *s)
{
if (fprintf(output,
"lines=%" PRIu64 "\nvalid=%" PRIu64 "\nmalformed=%" PRIu64 "\n"
"INFO=%" PRIu64 "\nWARN=%" PRIu64 "\nERROR=%" PRIu64 "\n"
"total_ms=%" PRIu64 "\n",
s->lines, s->valid, s->malformed,
s->by_level[LEVEL_INFO], s->by_level[LEVEL_WARN],
s->by_level[LEVEL_ERROR], s->total_ms) < 0) {
return false;
}
if (s->valid == 0) {
if (fputs("min_ms=n/a\nmax_ms=n/a\nmean_ms=n/a\n", output) == EOF) {
return false;
}
} else {
double mean = (double)s->total_ms / (double)s->valid;
if (fprintf(output, "min_ms=%u\nmax_ms=%u\nmean_ms=%.3f\n",
s->min_ms, s->max_ms, mean) < 0) {
return false;
}
}
return fflush(output) == 0;
}
The static assertion connects the input cap to representable duration sums at compile time.
stats_accept handles first-record extrema then updates the related counters and sum together.
stats_print uses PRIu64 for portable uint64_t formatting, avoids empty-data division, and flushes output before reporting success.
CLI, orchestration, and cleanup
Keep these file names and folders when building. Long lines scroll within the code panel.
/* Leon | Original teaching project. */
#include "reader.h"
#include "record.h"
#include "stats.h"
#include <inttypes.h>
#include <stdio.h>
#include <string.h>
enum exit_status { EXIT_CLEAN = 0, EXIT_IO = 1, EXIT_USAGE = 2, EXIT_DATA = 3 };
#define WARNING_LIMIT 5U
static void warn_bad_line(const struct stats *s)
{
if (s->malformed <= WARNING_LIMIT) {
fprintf(stderr, "warning: line %" PRIu64 ": malformed record\n", s->lines);
} else if (s->malformed == WARNING_LIMIT + 1U) {
fputs("warning: further malformed lines omitted\n", stderr);
}
}
/* Borrow input and caller-owned statistics. Report only complete input. */
static bool analyze(FILE *input, struct stats *s)
{
struct line_reader reader = {input, 0};
char line[MAX_LINE_BYTES + 1U];
for (;;) {
enum line_status status = read_line(&reader, line);
if (status == LINE_END) {
return true;
}
if (status == LINE_IO_ERROR) {
fputs("error: cannot read input\n", stderr);
return false;
}
if (status == LINE_LIMIT) {
fprintf(stderr, "error: input exceeds %u bytes\n", MAX_INPUT_BYTES);
return false;
}
struct record record;
if (status == LINE_BAD || !parse_record(line, &record)) {
stats_reject(s);
warn_bad_line(s);
} else {
stats_accept(s, &record);
}
}
}
int main(int argc, char **argv)
{
if (argc != 2) {
fputs("usage: log_analyzer FILE|-\n", stderr);
return EXIT_USAGE;
}
bool owns_input = strcmp(argv[1], "-") != 0;
FILE *input = owns_input ? fopen(argv[1], "rb") : stdin;
if (input == NULL) {
fprintf(stderr, "error: cannot open input: %s\n", argv[1]);
return EXIT_IO;
}
struct stats totals = {0};
bool completed = analyze(input, &totals);
if (owns_input && fclose(input) != 0) {
fputs("error: cannot close input\n", stderr);
completed = false;
}
if (!completed) {
return EXIT_IO;
}
if (!stats_print(stdout, &totals)) {
fputs("error: cannot write report\n", stderr);
return EXIT_IO;
}
return totals.malformed == 0 ? EXIT_CLEAN : EXIT_DATA;
}
main validates argument count and distinguishes owned file input from borrowed stdin.
analyze maps each reader result to parsing, rejection, clean completion, or fatal failure. Short-circuit || prevents parsing a LINE_BAD buffer.
Warning output is bounded separately from counting. Cleanup happens before printing, and the final return communicates data quality.
Reproducible multi-file build
Keep these file names and folders when building. Long lines scroll within the code panel.
# Leon | Original teaching project.
CC = clang
CFLAGS = -std=c11 -Wall -Wextra -Werror -pedantic -pthread
OBJECTS = main.o reader.o record.o stats.o
.PHONY: all test clean
all: log_analyzer
log_analyzer: $(OBJECTS)
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $(OBJECTS) $(LDLIBS)
main.o: main.c reader.h record.h stats.h
reader.o: reader.c reader.h
record.o: record.c record.h
stats.o: stats.c stats.h record.h reader.h
%.o: %.c
$(CC) $(CPPFLAGS) $(CFLAGS) -c $< -o $@
test: log_analyzer
python3 test.py
clean:
rm -f log_analyzer $(OBJECTS)
all names the executable. Four object files correspond to four C implementation files.
Header prerequisites make dependents rebuild when an interface changes. The pattern rule compiles; the executable rule links.
test first builds and then runs python3 test.py. clean removes only the named binary and objects in this project.
Valid synthetic robot records
Keep these file names and folders when building. Long lines scroll within the code panel.
INFO 10
WARN 40
INFO 20
ERROR 100
INFO 0
WARN 30
All six rows match the grammar, including a legitimate zero duration and an ERROR event.
Compute 10 + 40 + 20 + 100 + 0 + 30 = 200 before running the program.
Use this small fixture to compare terminal output with the literal expected report in test.py.
Valid data around three bad rows
Keep these file names and folders when building. Long lines scroll within the code panel.
INFO 5
DEBUG 8
WARN 10
ERROR -1
INFO 60001
ERROR 15
Rows 2, 4, and 5 respectively violate level, sign, and maximum-duration rules.
The remaining durations 5, 10, 15 sum to 30; all are positive, so they check first-record minimum initialization.
Expect exit 3 alongside a complete report and warnings, demonstrating that report availability and clean success differ.
A truly empty stream
Keep these file names and folders when building. Long lines scroll within the code panel.
Invisible-byte view (hexadecimal): empty file; zero bytes
This file has zero bytes. It contains neither a header nor a newline.
Its role is to test successful completion with zero observations, not malformed input.
Compare with the test that sends blank lines: those do create physical lines and rejection counts.
Executable behavior tests
Keep these file names and folders when building. Long lines scroll within the code panel.
# Leon | Original teaching project.
"""Behavior tests: run `make test`; every generated input stays in a temp dir."""
from pathlib import Path
import shutil
import subprocess
import tempfile
import unittest
ROOT = Path(__file__).resolve().parent
PROGRAM = ROOT / "log_analyzer"
class AnalyzerTests(unittest.TestCase):
def setUp(self):
self.temp = tempfile.TemporaryDirectory(prefix="log-analyzer-test-")
self.addCleanup(self.temp.cleanup)
self.work = Path(self.temp.name)
def run_log(self, data=b"", args=None):
if args is None:
args = ["-"]
self.assertTrue(PROGRAM.is_file(), "Build log_analyzer with make first")
return subprocess.run(
[str(PROGRAM), *args], input=data, capture_output=True,
cwd=self.work, timeout=5, check=False,
)
def fields(self, result):
lines = result.stdout.decode("ascii").splitlines()
self.assertEqual(len(lines), 10, result.stdout)
return dict(line.split("=", 1) for line in lines)
def test_demo_has_hand_calculated_totals(self):
shutil.copyfile(ROOT / "fixtures/demo.log", self.work / "demo.log")
result = self.run_log(args=["demo.log"])
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(result.stderr, b"")
self.assertEqual(result.stdout, (
b"lines=6\nvalid=6\nmalformed=0\nINFO=3\nWARN=2\nERROR=1\n"
b"total_ms=200\nmin_ms=0\nmax_ms=100\nmean_ms=33.333\n"
))
def test_mixed_file_keeps_good_rows_and_reports_exit_3(self):
shutil.copyfile(ROOT / "fixtures/mixed.log", self.work / "mixed.log")
result = self.run_log(args=["mixed.log"])
self.assertEqual(result.returncode, 3)
self.assertEqual(result.stdout, (
b"lines=6\nvalid=3\nmalformed=3\nINFO=1\nWARN=1\nERROR=1\n"
b"total_ms=30\nmin_ms=5\nmax_ms=15\nmean_ms=10.000\n"
))
for number in (2, 4, 5):
self.assertIn(f"line {number}: malformed record".encode(), result.stderr)
def test_empty_has_no_invented_latency(self):
result = self.run_log()
self.assertEqual(result.returncode, 0)
self.assertEqual(result.stdout, (
b"lines=0\nvalid=0\nmalformed=0\nINFO=0\nWARN=0\nERROR=0\n"
b"total_ms=0\nmin_ms=n/a\nmax_ms=n/a\nmean_ms=n/a\n"
))
def test_duration_boundaries(self):
result = self.run_log(b"INFO 0\nERROR 60000\n")
self.assertEqual(result.returncode, 0)
fields = self.fields(result)
self.assertEqual(fields["total_ms"], "60000")
self.assertEqual(fields["min_ms"], "0")
self.assertEqual(fields["max_ms"], "60000")
self.assertEqual(fields["mean_ms"], "30000.000")
def test_integer_grammar_rejects_signs_suffixes_and_overflow(self):
bad = [b"", b"+1", b"-1", b"1.5", b"1ms", b"1 2", b"60001",
b"999999999999999999999999", b"0x10", b"1e2"]
for token in bad:
with self.subTest(token=token):
result = self.run_log(b"INFO " + token + b"\n")
self.assertEqual(result.returncode, 3)
fields = self.fields(result)
self.assertEqual(fields["valid"], "0")
self.assertEqual(fields["malformed"], "1")
self.assertEqual(fields["mean_ms"], "n/a")
def test_level_must_match_whole_uppercase_word(self):
for level in (b"info", b"DEBUG", b"INF", b"INFOx", b"WARN3"):
with self.subTest(level=level):
result = self.run_log(level + b" 2\n")
self.assertEqual(result.returncode, 3)
self.assertEqual(self.fields(result)["valid"], "0")
def test_crlf_tabs_leading_zero_and_unterminated_last_line(self):
result = self.run_log(b" \tINFO\t0007 \r\nWARN 3")
self.assertEqual(result.returncode, 0)
fields = self.fields(result)
self.assertEqual(fields["lines"], "2")
self.assertEqual(fields["total_ms"], "10")
self.assertEqual(fields["mean_ms"], "5.000")
def test_line_limit_is_exact_and_drains_one_bad_record(self):
exact = b"INFO " + b"0" * 121 + b"1"
too_long = b"INFO 2" + b" " * 122
self.assertEqual(len(exact), 127)
self.assertEqual(len(too_long), 128)
result = self.run_log(exact + b"\n" + too_long + b"\nWARN 7\n")
self.assertEqual(result.returncode, 3)
fields = self.fields(result)
self.assertEqual(fields["lines"], "3")
self.assertEqual(fields["malformed"], "1")
self.assertEqual(fields["valid"], "2")
self.assertEqual(fields["total_ms"], "8")
def test_binary_bytes_do_not_hide_suffix_or_imitate_eof(self):
result = self.run_log(b"INFO 2\x00hidden\n\xff\nERROR 9\n")
self.assertEqual(result.returncode, 3)
fields = self.fields(result)
self.assertEqual(fields["lines"], "3")
self.assertEqual(fields["malformed"], "2")
self.assertEqual(fields["total_ms"], "9")
def test_blank_lines_count_and_diagnostics_are_capped(self):
result = self.run_log(b"\n" * 20)
self.assertEqual(result.returncode, 3)
self.assertEqual(self.fields(result)["malformed"], "20")
self.assertEqual(result.stderr.count(b"malformed record"), 5)
self.assertEqual(result.stderr.count(b"further malformed lines omitted"), 1)
def test_mean_uses_fractional_division(self):
result = self.run_log(b"INFO 1\nINFO 2\nINFO 2\n")
self.assertEqual(result.returncode, 0)
self.assertEqual(self.fields(result)["mean_ms"], "1.667")
def test_byte_budget_accepts_exactly_one_mebibyte(self):
row = b"INFO 0" + b" " * 121 + b"\n"
result = self.run_log(row * 8192)
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(self.fields(result)["valid"], "8192")
self.assertEqual(self.fields(result)["total_ms"], "0")
def test_byte_budget_failure_has_no_partial_report(self):
row = b"INFO 0" + b" " * 121 + b"\n"
result = self.run_log(row * 8192 + b"x")
self.assertEqual(result.returncode, 1)
self.assertEqual(result.stdout, b"")
self.assertIn(b"input exceeds 1048576 bytes", result.stderr)
def test_argument_count_is_a_usage_error(self):
for args in ([], ["one", "two"]):
with self.subTest(args=args):
result = self.run_log(args=args)
self.assertEqual(result.returncode, 2)
self.assertEqual(result.stdout, b"")
self.assertIn(b"usage:", result.stderr)
def test_missing_file_is_an_io_error(self):
result = self.run_log(args=["missing.log"])
self.assertEqual(result.returncode, 1)
self.assertEqual(result.stdout, b"")
self.assertIn(b"cannot open", result.stderr)
def test_directory_is_not_a_successful_input(self):
result = self.run_log(args=[str(self.work)])
self.assertEqual(result.returncode, 1)
self.assertEqual(result.stdout, b"")
self.assertTrue(b"cannot open" in result.stderr or
b"cannot read" in result.stderr)
if __name__ == "__main__":
unittest.main(verbosity=2)
setUp creates a fresh temporary directory; addCleanup removes it. run_log calls the real executable without a shell and with a timeout.
Fixtures have literal, independently calculated expected output. Other tests assert one targeted failure mode or boundary.
Both sides of limits are exercised. Missing-file and directory cases use real temporary paths; output and close failures are not exhaustively simulated.
Complete English run guide
Keep these file names and folders when building. Long lines scroll within the code panel.
# Streaming log analyzer
Leon | Original teaching project for COMP2017. [中文版](README.zh.md).
Leon is testing a small classroom robot simulator. After each simulated action,
it writes a severity and a duration, such as `WARN 40`. Reading the whole trace by
eye makes it easy to miss a slow action. Leon wants a tool that counts each
severity and summarizes durations. Some records are broken, so the report must
also say how much data was rejected. All fixtures are invented teaching data.
This is an original engineering project, not a supplied course assignment.
## Build, run, test
You need Clang, make, and Python 3 on macOS or Linux. From the repository root:
```sh
cd projects/comp2017-log-analyzer
make
./log_analyzer fixtures/demo.log
make test
```
For a downloaded project folder, open a terminal in that folder and start with
`make`. The Makefile compiles four C files separately and links them. Its default
flags are `-std=c11 -Wall -Wextra -Werror -pedantic -pthread`; this project creates
no threads. `make clean` removes only the executable and its object files.
To analyze standard input, use `-` as the only argument:
```sh
printf 'INFO 12\n' | ./log_analyzer -
```
The tool prints ten `key=value` lines. `total_ms` is the sum of valid durations;
`mean_ms = total_ms / valid`, rounded for display to three decimal places. The
minimum and maximum also use only valid rows. Empty data has `n/a` latency values:
no observation is different from an observation with a duration of zero.
## Agree on the input before writing the loop
A record is an uppercase `INFO`, `WARN`, or `ERROR`, then one or more spaces or
tabs, then decimal digits representing **0 through 60000 milliseconds**. Leading
and trailing spaces/tabs and leading zeroes are allowed. Signs, fractions, units,
extra columns, comments, and unknown levels are rejected. A blank line is one
malformed record. `ERROR` is a valid severity; it is not a parser error.
LF ends a physical line. A final CR is removed, which accepts CRLF files and a
final CR before EOF. The last record does not need LF. At most **127 bytes before
LF**, including a final CR, are allowed. An embedded NUL or longer line rejects
the entire physical line. The reader consumes its remaining bytes so they cannot
be mistaken for another row. The whole input may contain at most **1,048,576
bytes**, including line endings. One extra byte may be read to detect excess.
| Exit | Meaning | stdout |
|---|---|---|
| 0 | Input completed; every record was valid, or input was empty | Complete summary |
| 3 | Input completed; at least one record was malformed | Summary of valid rows plus rejected count |
| 2 | Wrong number of arguments | Empty |
| 1 | Open/read/close failure, byte budget exceeded, or reported output failure | No report for input failures; an output failure may leave a prefix |
Warnings go to stderr, with physical line numbers for the first five bad rows.
One further message says later warnings were omitted; all bad rows are still
counted. Run `echo $?` immediately after a command to see its exit status.
## Five learning milestones
1. **Follow one row through a working program.** Run the `INFO 12` command above.
Find `main`, `parse_record`, `stats_accept`, and `stats_print`. The text becomes
a `struct record`, then changes a `struct stats`, then becomes output text.
Draw those four boxes before reading error branches. The checkpoint is
`valid=1`, `INFO=1`, `total_ms=12`, and `mean_ms=12.000`.
2. **Make a parser promise.** Read `record.h` before `record.c`. A successful call
assigns the caller's record; failure leaves it unchanged. The local
`candidate` makes that promise possible. Trace `INFO 60000`, then `INFO 60001`.
Before computing `value * 10 + digit`, the code checks
`value <= (60000 - digit) / 10`. This both enforces the range and avoids an
overflowing intermediate. Confirm that `INFO 12ms` is rejected even though
its beginning looks numeric. The checkpoint is clean separation between
valid severity `ERROR` and malformed text.
3. **Read a stream without losing line boundaries.** Read `reader.h`, then
`reader.c`. The buffer needs 128 bytes: 127 data bytes plus `\0`. Keep `fgetc`'s
return value in an `int` so EOF remains distinguishable from a byte. A bad
line is drained before the next call. Test 127 bytes, 128 bytes, and a valid
row after the oversized row. The checkpoint is exactly one rejection, with
the later valid record intact. The reader borrows `FILE *`; it never closes it.
4. **Accumulate a useful report.** Read `stats.h` and `stats.c`. Initialize the
struct with `{0}`. For the first accepted value, set both extrema from that
value; initializing minimum to zero alone would give wrong results for
positive durations. Use `uint64_t` for counts and sums, `unsigned` for a
bounded duration, and `double` only when computing the mean. Recalculate the
demo by hand: sum 200, count 6, min 0, max 100. The checkpoint is the exact
demo output below and the empty-input `n/a` output.
5. **Finish the command-line contract and prove it.** Revisit `main.c`. Mark every
path from `fopen` to `fclose`. A file opened here is closed here; borrowed stdin
is left to the runtime. Input errors suppress a misleading partial report.
Run `make test`, then read a test that deliberately rejects input. It must
inspect the exit code, report, and diagnostics. The checkpoint is all 16
tests passing, including the 1 MiB limit and the mixed-data exit status 3.
The website provides longer bilingual steps and extracts from these same final
files. Extracts are reading aids, not standalone programs. Build the full project.
These checkpoints are original concept checks, not answers to supplied exercises.
## File reading order
| File | Why it exists |
|---|---|
| `record.h`, `record.c` | Record shape and complete text-validation contract |
| `reader.h`, `reader.c` | Byte budget, line framing, and stream errors |
| `stats.h`, `stats.c` | State invariants, aggregates, and report formatting |
| `main.c` | Arguments, resource ownership, pipeline, and exit policy |
| `Makefile` | Rebuild the needed translation units and link once |
| `fixtures/demo.log` | Six valid rows with hand-calculable results |
| `fixtures/mixed.log` | Three valid and three invalid rows |
| `fixtures/empty.log` | Zero bytes, not one blank line |
| `test.py` | Sixteen tests against the executable using temporary directories |
Headers are interfaces: read their promises first. Source files implement those
promises. No parser or statistics function stores a pointer to the line buffer;
it is safe to reuse the buffer on the next iteration. The main function owns the
statistics for its whole run. No application heap allocation is required.
## Three actual runs
These stdout blocks were captured from the compiled program. Build once with
`make` before running them. stderr is described separately.
### Valid robot trace
```sh
./log_analyzer fixtures/demo.log
```
```text
lines=6
valid=6
malformed=0
INFO=3
WARN=2
ERROR=1
total_ms=200
min_ms=0
max_ms=100
mean_ms=33.333
```
Exit 0; stderr is empty. `ERROR=1` counts a simulated event, not a program failure.
### A trace with damaged records
```sh
./log_analyzer fixtures/mixed.log
```
```text
lines=6
valid=3
malformed=3
INFO=1
WARN=1
ERROR=1
total_ms=30
min_ms=5
max_ms=15
mean_ms=10.000
```
Exit 3. stderr warns about lines 2, 4, and 5: unknown `DEBUG`, negative `-1`, and
out-of-range `60001`. Their durations never reach the aggregates.
### No observations
```sh
./log_analyzer fixtures/empty.log
```
```text
lines=0
valid=0
malformed=0
INFO=0
WARN=0
ERROR=0
total_ms=0
min_ms=n/a
max_ms=n/a
mean_ms=n/a
```
Exit 0; stderr is empty. An empty file has no malformed record.
## Debug with a prediction
If mean `1.667` becomes `1.000` for durations 1, 2, 2, inspect the division's types:
converting after integer division is too late. If a valid row after a long row is
lost or counted twice, inspect draining at the reader boundary. If all positive
inputs report minimum zero, inspect first-record initialization. If a file appears
successful after a read failure, inspect `ferror`, not just `EOF`. Start with the
smallest input that demonstrates the problem, and keep it as a regression test.
Tests cover valid/mixed/empty input; numeric and severity grammar; 0 and 60000;
CRLF, tabs, leading zeroes, and missing final LF; 127/128-byte rows; NUL and 0xff;
warning suppression; fractional mean; exact/excessive byte budgets; usage; missing
files; and directories. Expected totals are literal, hand-checked values, not a
second copy of the C algorithm. Every invocation has a five-second test timeout.
Generated files live only in each test's temporary directory and are removed.
The suite does not simulate every filesystem, output, or close failure.
## Decisions, limits, and next steps
Memory used by our state is O(1); time is O(B) for B bytes consumed. Standard I/O
also maintains its own implementation-managed buffers. The byte cap limits work
and proves the total fits: at most 1,048,576 accepted records times 60,000 ms is
62,914,560,000, safely below the `uint64_t` maximum. The actual accepted-record
limit is smaller because records use multiple bytes. Changing limits requires
reviewing that proof; a static assertion checks its multiplication bound.
We choose a small explicit grammar and a fixed buffer to make contracts visible.
This is not a parser for arbitrary real-world logs. There are no timestamps,
percentiles, live file following, or per-level latency summaries. A byte limit is
not a wall-clock deadline: stdin may wait for its producer. Use completed files
for the first project. A broken output pipe may trigger the normal POSIX SIGPIPE
signal rather than our exit 1 path. File read/write/close errors returned by the C
library are checked; failures writing diagnostic stderr are not retried.
To extend the project, add per-level duration sums and means. Acceptance: the demo
must give INFO 30/3 = 10.000, WARN 70/2 = 35.000, ERROR 100/1 = 100.000, and an absent
level must print `n/a`. Alternatively add a `--strict` mode. Acceptance: the first
bad row stops processing with exit 3 and no stdout, while default mode remains
unchanged. Write the new behavior tests before changing the implementation.
## Primary references
- [WG14 N1570: C11 committee draft](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf): sections 7.20, 7.21.7.1, and 7.21.10 explain integer types, `fgetc`, and stream status.
- [Clang user manual](https://clang.llvm.org/docs/UsersManual.html): language and diagnostic options.
- [Python subprocess documentation](https://docs.python.org/3/library/subprocess.html): argument lists, captured output, return codes, and timeouts.
Start with build/run/test commands and the exit-status table.
Use the five milestones and file order as your map through the source.
Reproduce the three captured examples, then read the tradeoffs and extension acceptance criteria.
Complete Chinese run guide
Keep these file names and folders when building. Long lines scroll within the code panel.
# 流式日志分析器
Leon | 为 COMP2017 编写的原创教学项目。[English](README.md)。
Leon 正在测试课堂用的小机器人模拟器。每完成一个模拟动作，它就记录一个
级别和耗时，例如 `WARN 40`。靠眼睛看完整份记录，很容易漏掉慢动作。因此我们
制作一个工具：统计各级别的次数，再计算耗时总和、最小值、最大值和平均值。
有些记录可能损坏，报告必须说明丢弃了多少行。所有示例数据均为人工构造。
这是原创工程项目，不是课程提供的作业或答案。
## 编译、运行和测试
在 macOS 或 Linux 上需要 Clang、make 和 Python 3。从仓库根目录执行：
```sh
cd projects/comp2017-log-analyzer
make
./log_analyzer fixtures/demo.log
make test
```
如果下载的是独立项目文件夹，请在该文件夹内打开终端，从 `make` 开始。
Makefile 分别编译四个 C 文件，然后链接。默认参数为
`-std=c11 -Wall -Wextra -Werror -pedantic -pthread`；这个项目本身没有创建线程。
`make clean` 只删除可执行文件和它的目标文件。
如果从标准输入读取，把唯一的参数写成 `-`：
```sh
printf 'INFO 12\n' | ./log_analyzer -
```
程序输出十行 `key=value`。`total_ms` 是有效记录的耗时总和；
`mean_ms = total_ms / valid`，显示时保留三位小数。最小值和最大值也只使用有效行。
没有有效记录时，三个耗时指标显示 `n/a`。没有观测值与观测到零毫秒是两回事。
## 先约定输入，再编写循环
每条记录先写大写的 `INFO`、`WARN` 或 `ERROR`，再写至少一个空格或制表符，
最后写表示 **0 到 60000 毫秒** 的十进制数字。允许前后空格、制表符以及数字前导零。
不接受正负号、小数、单位、多余字段、注释或未知级别。空白行算一条错误记录。
`ERROR` 是合法的事件级别，并不表示解析器出错。
LF 字节结束一条物理行。读取器去掉最后一个 CR，因此支持 CRLF，也接受文件末尾
换行缺失时的最后一个 CR。最后一条记录可以没有 LF。每行在 LF 之前最多有
**127 字节**，末尾的 CR 也占一个字节。出现嵌入的 NUL 或超长行时，整条物理行
作废。读取器继续吃掉这一行剩余的字节，避免把尾部误当成下一行。整个输入最多
**1,048,576 字节**，包含换行。为区分恰好到达文件末尾与超限，可能多读取一个字节。
| 退出码 | 含义 | stdout |
|---|---|---|
| 0 | 完成读取；所有行有效，或输入为空 | 完整报告 |
| 3 | 完成读取；至少有一条错误记录 | 有效行统计及错误行数 |
| 2 | 参数数量不正确 | 空 |
| 1 | 打开、读取、关闭失败，输入超限，或输出函数报告失败 | 输入失败时不输出报告；输出失败可能已写出一部分 |
警告写入 stderr。前五条错误显示物理行号，之后只提醒一次省略了更多警告。
错误行仍全部计数。要查看退出码，在一条命令执行后立刻运行 `echo $?`。
## 五个学习阶段
1. **让一行数据走完整条路径。** 运行上面的 `INFO 12` 命令。找到 `main`、
`parse_record`、`stats_accept` 和 `stats_print`。文本先变成 `struct record`，
再修改 `struct stats`，最后变成输出文本。先画出这四个方框，再阅读错误分支。
检查点是 `valid=1`、`INFO=1`、`total_ms=12` 和 `mean_ms=12.000`。
2. **给解析器写清楚承诺。** 先读 `record.h`，再读 `record.c`。调用成功才写入调用者
的记录，失败则保持它不变。局部变量 `candidate` 实现了这个承诺。手动跟踪
`INFO 60000` 和 `INFO 60001`。计算 `value * 10 + digit` 前，先检查
`value <= (60000 - digit) / 10`，同时控制范围和避免中间结果溢出。
确认 `INFO 12ms` 会被拒绝，即使前面看起来像整数。检查点是能够区分合法的
`ERROR` 级别和不合法的文本。
3. **连续读行，同时保住边界。** 先读 `reader.h`，再读 `reader.c`。缓冲区需要
128 字节：127 个数据字节加一个 `\0`。`fgetc` 的返回值用 `int` 保存，才能把
EOF 与普通字节区分开。坏行要先读到行尾，再读取下一条。测试 127 字节、128
字节，以及超长行之后的合法行。检查点是恰好拒绝一行，后续有效记录仍然保留。
读取器只是借用 `FILE *`，不负责关闭它。
4. **累积出有意义的报告。** 阅读 `stats.h` 和 `stats.c`。用 `{0}` 初始化结构体。
第一条有效记录要同时设置最大值和最小值；如果最小值只初始化为零，所有耗时
都为正数时就会算错。计数和总和用 `uint64_t`，有限范围的耗时用 `unsigned`，
计算平均值时才使用 `double`。手算示例：总和 200，数量 6，最小值 0，最大值
100。检查点是得到下方完全一致的示例输出，并让空输入显示 `n/a`。
5. **完成命令行规则，并用测试证明。** 回到 `main.c`，标出从 `fopen` 到 `fclose`
的每条路径。在这里打开的文件，在这里关闭；借来的 stdin 由运行时收尾。
输入错误时不能把不完整的统计伪装成完整报告。运行 `make test`，再读一个故意
输入错误数据的测试。它必须同时检查退出码、报告和诊断。检查点是全部 16 个
测试通过，其中包括 1 MiB 上限和混合数据的退出码 3。
网站提供更详细的双语步骤和来自这些最终文件的代码摘录。摘录用于阅读，不能单独
编译；请编译完整项目。这些检查点是原创概念练习，不是现成作业的答案。
## 文件阅读顺序
| 文件 | 作用 |
|---|---|
| `record.h`、`record.c` | 定义记录形状，约定并实现完整文本校验 |
| `reader.h`、`reader.c` | 控制字节总量、识别行边界并处理流错误 |
| `stats.h`、`stats.c` | 维护状态约束，计算统计并格式化报告 |
| `main.c` | 处理参数、资源归属、流水步骤和退出规则 |
| `Makefile` | 重新编译需要更新的编译单元，再链接 |
| `fixtures/demo.log` | 六行有效记录，可以手算 |
| `fixtures/mixed.log` | 三行有效记录和三行错误记录 |
| `fixtures/empty.log` | 零字节文件，不是一个空白行 |
| `test.py` | 在临时目录中测试真实程序，共十六项测试 |
头文件是接口，先读它的承诺；源文件负责实现承诺。解析和统计函数都不保存行缓冲区
的指针，因此下一次循环可以安全地重用缓冲区。主函数拥有整个运行期间的统计结构体。
应用程序自身不需要堆分配。
## 三次真实运行
以下 stdout 均来自实际编译后的程序。请先用 `make` 编译一次。stderr 单独说明。
### 完整的机器人轨迹
```sh
./log_analyzer fixtures/demo.log
```
```text
lines=6
valid=6
malformed=0
INFO=3
WARN=2
ERROR=1
total_ms=200
min_ms=0
max_ms=100
mean_ms=33.333
```
退出码 0，stderr 为空。`ERROR=1` 统计一次模拟事件，并非程序运行失败。
### 含有损坏记录的轨迹
```sh
./log_analyzer fixtures/mixed.log
```
```text
lines=6
valid=3
malformed=3
INFO=1
WARN=1
ERROR=1
total_ms=30
min_ms=5
max_ms=15
mean_ms=10.000
```
退出码 3。stderr 警告第 2、4、5 行：分别是未知级别 `DEBUG`、负数 `-1`，以及
超出范围的 `60001`。这些行的耗时不会进入统计。
### 没有观测记录
```sh
./log_analyzer fixtures/empty.log
```
```text
lines=0
valid=0
malformed=0
INFO=0
WARN=0
ERROR=0
total_ms=0
min_ms=n/a
max_ms=n/a
mean_ms=n/a
```
退出码 0，stderr 为空。空文件没有错误行。
## 先作预测，再调试
如果耗时 1、2、2 的平均值从 `1.667` 变成 `1.000`，检查除法两侧的类型。
整数除法结束后再转浮点数已经来不及。如果超长行后面的有效行丢失或重复，检查
读取器是否真正消费到行尾。如果全是正耗时却报告最小值零，检查第一条记录的
初始化。如果读取失败后仍报告成功，检查 `ferror`，不要只检查 EOF。
先制作能复现问题的最小输入，再把它留下作为回归测试。
测试覆盖有效、混合和空输入；数字和级别语法；0 和 60000；CRLF、制表符、
前导零和缺失的末尾 LF；127/128 字节行；NUL 和 0xff；警告数量限制；小数平均值；
恰好达到和超过字节上限；参数错误；不存在的文件和目录。预期总和是手算的常量，
并非把 C 算法再写一遍。每次程序调用都有五秒测试超时。生成的输入只保存在每项
测试自己的临时目录中，结束后删除。测试没有模拟所有文件系统、输出或关闭故障。
## 取舍、限制与后续练习
我们维护的状态使用 O(1) 内存，处理 B 个输入字节的时间为 O(B)。标准 I/O 自己
还会维护由实现管理的缓冲区。字节上限控制工作量，也给出总和不会溢出的证明：
即使最多 1,048,576 条记录，每条 60,000 毫秒，总和也只有 62,914,560,000，
远小于 `uint64_t` 的上限。实际能容纳的记录更少，因为每条记录占多个字节。
改变上限时要重新检查这个证明；静态断言会检查乘法边界。
小而明确的语法和固定缓冲区便于理解接口。这不是通用真实日志格式解析器，
也没有时间戳、百分位数、实时跟踪文件或各级别独立耗时统计。字节限制不是运行
时间限制，stdin 仍可能等待数据生产者。第一个项目建议读取已经写完的普通文件。
输出管道断开时，POSIX 默认的 SIGPIPE 信号可能终止程序，而不经过退出码 1 的
路径。C 库返回的读、写和关闭错误会检查；向诊断流 stderr 写入失败时不会重试。
后续可以增加每个级别的耗时总和与均值。验收：示例中 INFO 为 30/3 = 10.000，
WARN 为 70/2 = 35.000，ERROR 为 100/1 = 100.000；未出现的级别显示 `n/a`。
也可以增加 `--strict` 模式。验收：首次遇到坏行就停止，退出码 3，stdout 为空；
默认模式保持不变。先写描述新行为的测试，再修改实现。
## 官方原始参考资料
- [WG14 N1570：C11 委员会草案](https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1570.pdf)：7.20、7.21.7.1 和 7.21.10 节分别介绍整数类型、`fgetc` 与流状态。
- [Clang 用户手册](https://clang.llvm.org/docs/UsersManual.html)：语言标准与诊断选项。
- [Python subprocess 文档](https://docs.python.org/3/library/subprocess.html)：参数列表、捕获输出、返回码与超时。
The Chinese guide carries the same commands and input contract as the English guide.
Use its type, ownership, and debugging explanations alongside the English source identifiers.
Output keys remain stable English strings in both languages, so the executable and tests do not change with the guide language.
First add hand-calculated tests. Store a count and sum per level, update them only after successful parsing, then print each mean with the same empty-level policy.
The demo yields INFO mean 10.000, WARN mean 35.000, ERROR mean 100.000. A missing level prints n/a, malformed rows never contribute, and old global metrics remain unchanged.
Extend the argument contract with --strict. Decide the policy before coding: stop on the first malformed row, close any owned stream, and omit stdout. Keep the current default behavior.
The mixed fixture in strict mode returns 3, warns only about line 2, and prints no report. Without the flag its existing output is unchanged; clean and empty inputs still return 0.