Add an option for copying comments in a header file over to the generated bindings

[xpple]

This PR adds an option for copying comments in a header file over to the generated bindings. See also my previous email.

The option can be enabled by passing the --copy-comments flag to jextract. This flag is optional, and false by default. This means that if the flag is not passed, the output from jextract will be identical to what it was before this change.

I decided against using the existing clang API for Doxygen comments, because they are far from standardized in C projects. This meant though that I had to come up with custom logic for associating comments with declarations. I am happy to incorporate feedback on how this association can be improved. Currently, all comments that immediately precede a declaration are associated with that declaration. Whitespace is ignored.

For example, in the below code, both comments are associated with the func declaration.

// one
// two
void func(void);

For structs/unions, comments that precede member declarations will be associated with that member.

/**
 * will be associated with Point
 */
struct Point {
    /// will be associated
    /// with x
    int x;
    /// will be associated
    /// with y
    int y;
};

Similarly, enum constants can have comments too. Comments at the enum tag are ignored, because the enum tag itself is not retained in the jextract bindings.

// this comment is ignored
enum Color {
    // will be associated with RED
    RED,
    // will be associated with GREEN
    GREEN,
    // will be associated with BLUE
    BLUE,
};

In the API, the Declaration interface now has a List<String> comments() method. These are the raw comments, so they still contain the comment delimiters (// or /* ... */). If --copy-comments is not passed, this method will return an empty list (not null).

In the binding generation code, I made an attempt to detect some common styles for comments, and account for them separately. For example, if a comment starts with three forward slashes, all three will be stripped (and not just the two that lexically define the comment). The code for this is in ClassSourceBuilder#copyComments. Below is what the JavaDoc will look like for the above three examples.

func

/**
 * {@snippet lang=c :
 * void func()
 * }
 * <p><strong>Copied comments:</strong></p>
 * one
 * two
 */
public static void func() {
    // ...
}

Color

/**
 * {@snippet lang=c :
 * enum Color.RED = 0
 * }
 * <p><strong>Copied comments:</strong></p>
 * will be associated with RED
 */
public static int RED() {
    // ...
}

/**
 * {@snippet lang=c :
 * enum Color.GREEN = 1
 * }
 * <p><strong>Copied comments:</strong></p>
 * will be associated with GREEN
 */
public static int GREEN() {
    // ...
}

/**
 * {@snippet lang=c :
 * enum Color.BLUE = 2
 * }
 * <p><strong>Copied comments:</strong></p>
 * will be associated with BLUE
 */
public static int BLUE() {
    // ...
}

Point

/**
 * {@snippet lang=c :
 * struct Point {
 *     int x;
 *     int y;
 * }
 * }
 * <p><strong>Copied comments:</strong></p>
 * will be associated with Point
 */
public class Point {
    // ...

    /**
     * Getter for field:
     * {@snippet lang=c :
     * int x
     * }
     * <p><strong>Copied comments:</strong></p>
     * will be associated
     * with x
     */
    public static int x(MemorySegment struct) {
        // ...
    }

    /**
     * Setter for field:
     * {@snippet lang=c :
     * int x
     * }
     * <p><strong>Copied comments:</strong></p>
     * will be associated
     * with x
     */
    public static void x(MemorySegment struct, int fieldValue) {
        // ...
    }

    // ...

    /**
     * Getter for field:
     * {@snippet lang=c :
     * int y
     * }
     * <p><strong>Copied comments:</strong></p>
     * will be associated
     * with y
     */
    public static int y(MemorySegment struct) {
        // ...
    }

    /**
     * Setter for field:
     * {@snippet lang=c :
     * int y
     * }
     * <p><strong>Copied comments:</strong></p>
     * will be associated
     * with y
     */
    public static void y(MemorySegment struct, int fieldValue) {
        // ...
    }

    // ...
}

Not all JavaDoc comments pertaining to a certain declaration get the comments associated with that declaration. For example, for functions the $descriptor, $handle and $address methods do not get the comments, and for structs the $layout and $offset methods do not get them. The documentation is usually only relevant for calling/accessing/modifying something, not for derived properties.

The comment association code works by keeping track of the previous cursor's extent's ending, and tokenizing between that and the current cursor's extent's beginning. For example, if in the below code the current cursor is func2, then the tokens encountered will be ;, // comment and void.

void func1(void);

// comment
void func2(void);

These tokens will be iterated over in reverse order, ignoring the first token(s) (here only void) as they are part of the current declaration. Then, as long as the encountered token is a comment (CXToken_Comment), it will be associated with the declaration. In cases where this approach can't work (e.g. there is no previous declaration, the previous declaration is in a different file or the previous declaration actually appears after the current declaration lexically), the code will resort to a fallback.

Since I needed clang_getRange, which wasn't in the symbols list (updateclang/clang.symbols) yet, I had to regenerate the bindings. Because jextract had received some updates since the last time the bindings were generated, there are some bindings changes unrelated to clang_getRange as well. While updating, I noticed the instructions had gotten slightly out of date, so I updated those as well. I added three patch files which people can git apply to apply the necessary changes more easily.

I made sure to maintain source and binary compatibility in the public API of jextract.

I did not test the PR locally, but the tests I added passed on GitHub Actions. As this is my first time working with libclang, I am happy receive feedback for this PR.

• I confirm that I make this contribution in accordance with the OpenJDK Interim AI Policy.

---

Progress

• Change must not contain extraneous whitespace
• Change must be properly reviewed (no review required)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jextract.git pull/308/head:pull/308
$ git checkout pull/308

Update a local copy of the PR:
$ git checkout pull/308
$ git pull https://git.openjdk.org/jextract.git pull/308/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 308

View PR using the GUI difftool:
$ git pr show -t 308

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jextract/pull/308.diff

Using Webrev

Link to Webrev Comment

[bridgekeeper]

👋 Welcome back xpple! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

[openjdk]

@xpple This change now passes all automated pre-integration checks.

After integration, the commit message for the final commit will be:

Add an option for copying comments in a header file over to the generated bindings

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

As you do not have Committer status in this project an existing Committer must agree to sponsor your change.

➡️ To flag this PR as ready for integration with the above commit message, type /integrate in a new comment. (Afterwards, your sponsor types /sponsor in a new comment to perform the integration).

[mlbridge]

Webrevs

• 01: Full - Incremental (7fecf1c3)
• 00: Full (abf52d20)

[xpple]

There is currently a bug where copied comments could contain */, ending the emitted JavaDoc comment and resulting in invalid Java code. This can be fixed by removing */ from the copied comments¹, but I think a better fix is to use Markdown comments instead. I have code ready locally that migrates jextract to use Markdown comments everywhere, but I'll hold back on pushing that code because this migration is probably something that should be discussed first.

For what it's worth, I think using Markdown comments is a great idea. It would break compatibility with JDK 22, but I don't think retaining that compatibility is a goal.

Footnotes

1. Note that trivial replacements fall victim to the classic nonidempotent sanitisation problem. E.g. the comment /// **// would still contain */ after removing */. ↩

[mlbridge]

Mailing list message from Jonathan Strauss on jextract-dev:

I did similar work on this in a private fork, I ran into a few traps, I can't recall what it was but just keep that in mind. If I have time I can dig some of it up.

On Sun, Aug 2, 2026, at 5:05 PM, Frederik van der Els wrote:

[xpple]

Please do! I believe everything to be working currently, but it's possible I missed something.