WIP: Tracking PR for initial parser #4

Good start. Biggest issues are the cursor pre-bump (missing comments at offset 0) and not advancing to end-of-line when a // comment isn't an apidoc comment, which can cause scanning inside the same comment line. Fixing those will make the parser more reliable.

Important

Parser misses comments at file start due to pre-bump: parse() currently skips the very first character of the input because the cursor is bumped at the start of the loop. This will miss comments at the start of the file. Consider changing the loop structure to avoid pre-incrementing before inspection.

Nice to have

Remove debug output from library: The parser emits println! and dbg! from a library crate, which will be noisy for callers and makes it hard to use in non-CLI contexts. Consider gating with a feature or removing once debugging is done.

Debug: raw JSON

{"file_comments":[{"body":"The loop advances the cursor before inspecting `slice`, which skips the very first character of the file. As a result, a comment that starts at byte 0 will never be detected. Consider checking `is_finished()` before bumping or moving the bump to the end of the loop.","line":17,"patch":"@@\n-    while !cursor.bump().is_finished() {\n-        let slice = cursor.slice();\n+    while !cursor.bump().is_finished() {\n+        let slice = cursor.slice();\n","path":"src/apidoc_parser/src/lib.rs","severity":"bug"},{"body":"When a `//` comment is not an apidoc comment, you `continue` without advancing to end-of-line. This leaves the cursor in the middle of the same line and can cause repeated scanning within the comment body (and potentially false positives if `//` appears again). You likely want to skip to the newline before continuing.","line":27,"patch":"@@\n-            if !cursor.slice().starts_with(APIDOC_PRAGMA) {\n-                continue;\n-            }\n+            if !cursor.slice().starts_with(APIDOC_PRAGMA) {\n+                continue;\n+            }\n","path":"src/apidoc_parser/src/lib.rs","severity":"important"}],"important":[{"body":"`parse()` currently skips the very first character of the input because the cursor is bumped at the start of the loop. This will miss comments at the start of the file. Consider changing the loop structure to avoid pre-incrementing before inspection.","title":"Parser misses comments at file start due to pre-bump"}],"nice_to_have":[{"body":"The parser emits `println!` and `dbg!` from a library crate, which will be noisy for callers and makes it hard to use in non-CLI contexts. Consider gating with a feature or removing once debugging is done.","title":"Remove debug output from library"}],"overall_comment":"Good start. Biggest issues are the cursor pre-bump (missing comments at offset 0) and not advancing to end-of-line when a `//` comment isn't an apidoc comment, which can cause scanning inside the same comment line. Fixing those will make the parser more reliable.","summary":"Parser loop skips first byte; non-apidoc comments aren't skipped to EOL, causing incorrect scanning."}

### Pull request overview **Parser loop skips first byte; non-apidoc comments aren't skipped to EOL, causing incorrect scanning.** _PR #4: WIP: Tracking PR for initial parser_ Good start. Biggest issues are the cursor pre-bump (missing comments at offset 0) and not advancing to end-of-line when a `//` comment isn't an apidoc comment, which can cause scanning inside the same comment line. Fixing those will make the parser more reliable. #### Important - **Parser misses comments at file start due to pre-bump**: `parse()` currently skips the very first character of the input because the cursor is bumped at the start of the loop. This will miss comments at the start of the file. Consider changing the loop structure to avoid pre-incrementing before inspection. #### Nice to have - **Remove debug output from library**: The parser emits `println!` and `dbg!` from a library crate, which will be noisy for callers and makes it hard to use in non-CLI contexts. Consider gating with a feature or removing once debugging is done. <details><summary>Debug: raw JSON</summary> ```json {"file_comments":[{"body":"The loop advances the cursor before inspecting `slice`, which skips the very first character of the file. As a result, a comment that starts at byte 0 will never be detected. Consider checking `is_finished()` before bumping or moving the bump to the end of the loop.","line":17,"patch":"@@\n- while !cursor.bump().is_finished() {\n- let slice = cursor.slice();\n+ while !cursor.bump().is_finished() {\n+ let slice = cursor.slice();\n","path":"src/apidoc_parser/src/lib.rs","severity":"bug"},{"body":"When a `//` comment is not an apidoc comment, you `continue` without advancing to end-of-line. This leaves the cursor in the middle of the same line and can cause repeated scanning within the comment body (and potentially false positives if `//` appears again). You likely want to skip to the newline before continuing.","line":27,"patch":"@@\n- if !cursor.slice().starts_with(APIDOC_PRAGMA) {\n- continue;\n- }\n+ if !cursor.slice().starts_with(APIDOC_PRAGMA) {\n+ continue;\n+ }\n","path":"src/apidoc_parser/src/lib.rs","severity":"important"}],"important":[{"body":"`parse()` currently skips the very first character of the input because the cursor is bumped at the start of the loop. This will miss comments at the start of the file. Consider changing the loop structure to avoid pre-incrementing before inspection.","title":"Parser misses comments at file start due to pre-bump"}],"nice_to_have":[{"body":"The parser emits `println!` and `dbg!` from a library crate, which will be noisy for callers and makes it hard to use in non-CLI contexts. Consider gating with a feature or removing once debugging is done.","title":"Remove debug output from library"}],"overall_comment":"Good start. Biggest issues are the cursor pre-bump (missing comments at offset 0) and not advancing to end-of-line when a `//` comment isn't an apidoc comment, which can cause scanning inside the same comment line. Fixing those will make the parser more reliable.","summary":"Parser loop skips first byte; non-apidoc comments aren't skipped to EOL, causing incorrect scanning."} ``` </details>

tommy requested changes 2026-01-29 17:39:41 +00:00

Dismissed

tommy left a comment

See summary comment above for details.

tommy commented

2026-01-29 17:39:41 +00:00

File: src/apidoc_parser/src/lib.rs:17 (bug)

The loop advances the cursor before inspecting slice, which skips the very first character of the file. As a result, a comment that starts at byte 0 will never be detected. Consider checking is_finished() before bumping or moving the bump to the end of the loop.

@@
-    while !cursor.bump().is_finished() {
-        let slice = cursor.slice();
+    while !cursor.bump().is_finished() {
+        let slice = cursor.slice();

**File:** src/apidoc_parser/src/lib.rs:17 (bug) The loop advances the cursor before inspecting `slice`, which skips the very first character of the file. As a result, a comment that starts at byte 0 will never be detected. Consider checking `is_finished()` before bumping or moving the bump to the end of the loop. ```diff @@ - while !cursor.bump().is_finished() { - let slice = cursor.slice(); + while !cursor.bump().is_finished() { + let slice = cursor.slice(); ```

tommy commented

2026-01-29 17:39:42 +00:00

File: src/apidoc_parser/src/lib.rs:27 (important)

When a // comment is not an apidoc comment, you continue without advancing to end-of-line. This leaves the cursor in the middle of the same line and can cause repeated scanning within the comment body (and potentially false positives if // appears again). You likely want to skip to the newline before continuing.

@@
-            if !cursor.slice().starts_with(APIDOC_PRAGMA) {
-                continue;
-            }
+            if !cursor.slice().starts_with(APIDOC_PRAGMA) {
+                continue;
+            }

**File:** src/apidoc_parser/src/lib.rs:27 (important) When a `//` comment is not an apidoc comment, you `continue` without advancing to end-of-line. This leaves the cursor in the middle of the same line and can cause repeated scanning within the comment body (and potentially false positives if `//` appears again). You likely want to skip to the newline before continuing. ```diff @@ - if !cursor.slice().starts_with(APIDOC_PRAGMA) { - continue; - } + if !cursor.slice().starts_with(APIDOC_PRAGMA) { + continue; + } ```

blueplum dismissed tommy's review 2026-01-29 17:46:20 +00:00

Reason:

This is a tracking PR.

tototomate123 commented

2026-01-29 18:10:04 +00:00

This is a tracking PR.

I get that this is a tracking / WIP PR, but I still think early feedback here is valuable.
Parser logic is pretty sensitive, and catching things like cursor movement and skipping behavior early can save us from baking in assumptions that are harder to unwind later. Even with just one commit, Tommy may have highlighted areas that would otherwise not be revisited later, which is why I still think this feedback is valuable.

> This is a tracking PR. I get that this is a tracking / WIP PR, but I still think early feedback here is valuable. Parser logic is pretty sensitive, and catching things like cursor movement and skipping behavior early can save us from baking in assumptions that are harder to unwind later. Even with just one commit, Tommy may have highlighted areas that would otherwise not be revisited later, which is why I still think this feedback is valuable.

This pull request is marked as a work in progress.

This branch is out-of-date with the base branch

View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.

git fetch -u origin feature/parser:feature/parser

git checkout feature/parser

Sign in to join this conversation.

3 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: blueplum/apidoc#4