diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8278ed7..f7baf9a 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -31,6 +31,45 @@ The following rule governs code contributions: - Contributions must be licensed under the [Apache 2.0 License](./LICENSE) - Due to legal reasons, contributors will be asked to accept a Developer Certificate of Origin (DCO) when they create the first pull request to this project. This happens in an automated fashion during the submission process. SAP uses [the standard DCO text of the Linux Foundation](https://developercertificate.org/). +## CI and Release + +### CI Checks + +Every pull request to `main` and every push to `main` runs the following checks (see [`.github/workflows/pipeline.yml`](.github/workflows/pipeline.yml)): + +| Job | What it enforces | +|---|---| +| `tests` | Unit tests (`mvn test -P '!with-integration-tests'`) on Java 17 and 21 | +| `integration-tests` | Full integration tests on Java 17 and 21 | +| `local-mtx-tests` | MTX lifecycle tests using a local sidecar (`mvn verify -pl integration-tests/mtx-local/srv`) on Java 17 and 21 | +| `sonarqube-scan` | SonarQube static analysis | +| `codeql` | CodeQL security scanning (`security-extended` queries) on Java/Kotlin and GitHub Actions | +| `blackduck` (main only) | BlackDuck full open-source compliance scan | + +Static analysis runs at **compile time on every build** via Maven plugins configured in the root `pom.xml`: + +- **Spotless** — Google Java Format + SAP license headers (enforced at `process-sources`) +- **SpotBugs** — `effort=Max` (at `process-test-classes`) +- **PMD + CPD** — SAP Cloud SDK ruleset (at `process-test-classes`) +- **Maven Enforcer** — no duplicate dependency versions, requires Maven 3.6.3+ and Java 21+ + +Integration tests live in [`integration-tests/`](integration-tests/README.md) — see that README for test modules, run commands, and profiles. + +### JaCoCo Coverage + +Coverage is aggregated in the `coverage-report/` module using JaCoCo's `report-aggregate` goal. Each feature module runs `prepare-agent` to collect `.exec` data; the `coverage-report` module merges all `.exec` files and generates a single HTML report fed into SonarQube. Coverage thresholds (80% on new code) are enforced by SonarQube in the pipeline. + +To generate the report locally: + +```bash +mvn clean verify +# Report: coverage-report/target/site/jacoco-aggregate/index.html +``` + +### Release + +Releases are triggered by publishing a GitHub Release with a matching tag. The [`release.yml`](.github/workflows/release.yml) workflow gates on two protected environments (`release-approval` then `release`), verifies the POM `revision` matches the tag, runs BlackDuck, builds without integration tests, and deploys to Maven Central with GPG signing. + ## Issues and Planning - We use GitHub issues to track bugs and enhancement requests. diff --git a/README.md b/README.md index b2ec6ec..933180c 100644 --- a/README.md +++ b/README.md @@ -4,7 +4,7 @@ ## About this project -This repository contains a collection of AI plugins for [CAP Java](https://cap.cloud.sap/docs/java/) applications, leveraging [SAP AI Core](https://help.sap.com/docs/sap-ai-core) and the SAP-RPT-1 foundation model. +This repository contains a collection of AI plugins for [CAP Java](https://cap.cloud.sap/docs/java/) applications, leveraging [SAP AI Core](https://help.sap.com/docs/sap-ai-core), the SAP-RPT-1 foundation model, and SAP Document AI. ### Plugins @@ -16,32 +16,17 @@ This repository contains a collection of AI plugins for [CAP Java](https://cap.c ### Starter -For the simplest setup, add the **`cds-starter-ai`** dependency which bundles both plugins: - -```xml - - com.sap.cds - cds-starter-ai - ${cds-ai.version} - -``` - -```json -"dependencies": { - "@cap-js/ai": "^1" -} -``` - -> Note: [`cds-feature-sap-document-ai`](cds-feature-sap-document-ai/README.md) is not part of `cds-starter-ai` yet - it will be added once the plugin supports multi-tenancy. Add it as an explicit dependency in your application if you need it today. +See [`cds-starter-ai`](cds-starter-ai/README.md) for the quickest setup. ## Prerequisites - Java 17+ - CAP Java 4.9+ - Node.js 20+ with `@sap/cds-dk` 9+ (for CDS build tooling) -- An [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding (for production use) +- An [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding (for `cds-feature-ai-core` and `cds-feature-recommendations`) +- A [SAP Document Information Extraction](https://help.sap.com/docs/document-ai) service binding (for `cds-feature-sap-document-ai`) -Without an AI Core binding the plugins fall back to mock implementations for local development. +Without the respective service binding, each plugin falls back to a mock or degraded mode for local development. ## Samples @@ -54,7 +39,7 @@ mvn clean install # build all modules mvn test # run unit tests ``` -For per-plugin details (configuration, programmatic API, multi-tenancy behaviour) see the individual module READMEs. For integration tests against a real AI Core instance see [`integration-tests/spring`](integration-tests/spring). +For per-plugin details (configuration, programmatic API, multi-tenancy behaviour) see the individual module READMEs. For integration tests against a real AI Core instance see [`integration-tests/`](integration-tests/README.md). ## Support, Feedback, Contributing diff --git a/cds-feature-recommendations/README.md b/cds-feature-recommendations/README.md index 03c0a6f..cb74aa5 100644 --- a/cds-feature-recommendations/README.md +++ b/cds-feature-recommendations/README.md @@ -2,7 +2,7 @@ AI-powered field recommendations for SAP Fiori UIs in CAP Java applications, leveraging SAP AI Core and the SAP-RPT-1 foundation model. -## How It Works +## Features The plugin generically hooks into any draft-enabled entity that has properties with a value help. When a user edits a draft record, the plugin: @@ -98,6 +98,11 @@ annotate Books with { }] } } + +// Opt out a specific field +annotate Books with { + genre @UI.RecommendationState: 0; +} ``` #### Adding the SAP_Recommendations navigation property manually diff --git a/cds-feature-sap-document-ai/README.md b/cds-feature-sap-document-ai/README.md index e7977ba..8f91c7a 100644 --- a/cds-feature-sap-document-ai/README.md +++ b/cds-feature-sap-document-ai/README.md @@ -9,6 +9,7 @@ A CAP Java plugin that integrates [SAP Document AI](https://help.sap.com/docs/do - [Integration Guide](#integration-guide) - [Usage](#usage) - [CDS Model](#cds-model) +- [Multi-Tenancy](#multi-tenancy) - [Bookshop Sample](#bookshop-sample) - [Running without a Document AI service binding](#running-without-a-die-service-binding) - [Running with a Document AI service binding (hybrid mode)](#running-with-a-die-service-binding-hybrid-mode) @@ -190,7 +191,9 @@ Submit a document via your application. The plugin logs progress at `INFO` level > **Note:** In the current version, document extraction can only be triggered programmatically via event emission, as shown in the [Integration Guide](#integration-guide). Annotation-based triggering (e.g. declaratively marking an entity field or action to trigger extraction) is not yet supported and is planned for a future release. -> **Note:** Multitenancy is not implemented in the current version and is planned for a future release. Until then, the plugin should only be used in single-tenant deployments. +## Multi-Tenancy + +Multi-tenancy is not implemented in the current version and is planned for a future release. The `tenantId` field is stored on the `ExtractionJob` entity as groundwork. ### CDS Model @@ -273,6 +276,8 @@ The plugin will resolve the Document AI service binding at startup, construct an The `AdminService` exposes a `Books` entity with a bound action `extractDocumentData()` illustrating how to trigger extraction from a CAP action. The `Attachments` composition on `Books` provides a Fiori UI for file upload and is used here purely as a convenient way to supply documents in the sample. The CAP Attachments plugin is not a dependency of this plugin - document storage and retrieval are outside the scope of `cds-feature-sap-document-ai`, which is concerned solely with submitting documents to SAP Document AI and delivering the extracted results. +A sample PDF invoice (`dummy invoice.pdf`) is included in the `samples/bookshop/` directory. You can upload it via the Fiori UI and trigger `extractDocumentData()` to see the full extraction flow end-to-end without needing your own test document. + --- ## How It Works @@ -382,6 +387,13 @@ The plugin communicates with the SAP Document Information Extraction service via --- +## Known Limitations + +- **Multi-tenancy** — not implemented; all jobs run in a single-tenant context. Planned for a future release. +- **Annotation-based triggering** — document extraction can only be initiated programmatically via event emission; declarative triggering is not yet supported. + +--- + ## Monitoring and Logging All plugin log statements are prefixed with `[sap-document-ai]` to facilitate log filtering. The plugin uses SLF4J and is configured through the standard logging framework of the host application. diff --git a/cds-feature-sap-document-ai/docs/handover.md b/cds-feature-sap-document-ai/docs/handover.md index f814cf6..91d377b 100644 --- a/cds-feature-sap-document-ai/docs/handover.md +++ b/cds-feature-sap-document-ai/docs/handover.md @@ -55,7 +55,6 @@ This is an alpha release. The core extraction pipeline works end-to-end, and the | S9 | **CLI scaffolding** | Setup is manual and documented in the README. A `cds add document-ai` command is not yet implemented. | | S10 | **OData API** | REST API works across all plans. OData support for higher-tier plans is a future enhancement. | | S11 | **Job cleanup** | Job records are kept for observability. A configurable retention policy is not yet implemented. | -| S12 | **Multiple bindings** | One Document AI binding is resolved at startup. Routing documents to different instances by type, region, or business unit is not yet supported. | --- @@ -79,26 +78,36 @@ If a binding is added after the app starts, it won't be picked up until a restar These are ideas and suggestions and not a fixed plan. The ordering reflects what felt most important at the time of writing, but the incoming team should feel free to reprioritise based on their own context and stakeholder needs. -### 1. Multitenancy _(S1)_ +### 1. Multitenancy _(S1)_ - Priority A natural first area to tackle would be making each tenant use its own Document AI credentials with isolated jobs. The polling logic and HTTP client would need to become tenant-aware - the `tenantId` field is already on `ExtractionJob`, so no schema migration is needed. -### 2. Annotation-Based Triggering _(S2)_ +**Tracking issue:** [#98](https://github.com/cap-java/cds-ai/issues/98) + +### 2. Annotation-Based Triggering _(S2)_ - Priority One possible enhancement is to allow developers to annotate a CDS entity field with `@DocumentAI` to automatically trigger extraction - removing the need for boilerplate event emission. This could cover both the backend (plugin reacts to annotated field writes) and the Fiori Elements UI (an "Upload & Extract" button injected automatically on the Object Page). +**Tracking issue:** [#97](https://github.com/cap-java/cds-ai/issues/97) + ### 3. Job Recovery on Startup _(S6)_ A useful addition could be a startup check for any jobs left in `PENDING`, `SUBMITTED`, or `RUNNING` status from before a restart, resuming polling for them automatically rather than waiting for a new submission to arrive. +**Tracking issue:** [#100](https://github.com/cap-java/cds-ai/issues/100) + ### 4. Extraction Progress Indicator The backend already tracks `SUBMITTED` and `RUNNING` states - it could be worth surfacing that status in the Fiori Elements Object Page as a visible progress indicator or status strip so users have feedback while extraction is running. +**Tracking issue:** [#108](https://github.com/cap-java/cds-ai/issues/108) + ### 5. Automatic Field Mapping _(S5)_ One idea is to have the plugin match extracted fields to CDS entity properties by name convention and pre-fill the Fiori form automatically. Fields below a configurable confidence threshold could be visually flagged (e.g. amber highlight) so users know what to double-check before saving. +**Tracking issue:** [#101](https://github.com/cap-java/cds-ai/issues/101) + ### 6. Document Viewer with Extraction Highlights and Human-in-the-Loop Verification A more exploratory idea is to visualise extracted fields on the document itself - bounding boxes colour-coded by confidence level, with click-to-focus between the document viewer and the form. Bounding box coordinates come back from Document AI and would need to be stored alongside the extraction result and exposed to a UI viewer component. @@ -107,45 +116,62 @@ It could also be worth visually marking AI-extracted fields in the Fiori form (e A further possibility is a human-in-the-loop confirmation step: the user reviews the extracted fields, corrects any errors, and explicitly confirms the result. This confirmed payload could be submitted back to Document AI as ground-truth feedback to activate the [instant learning](https://help.sap.com/docs/document-ai/sap-document-ai/instant-learning?locale=en-US) feature, improving model accuracy for that schema over time. +**Tracking issue:** [#109](https://github.com/cap-java/cds-ai/issues/109) + ### 7. Document AI Outbound Channels - Push-Based Result Delivery Document AI supports outbound channels at the schema level: notification channels (status pushes) and extension channels (callbacks triggered after prediction). One option worth exploring is registering the plugin as a target so Document AI pushes results to it directly, eliminating the need to poll. The `DocumentAiClient` interface and `ExtractionService.updateExtractionResult()` are already the right place to plug this in. +**Tracking issue:** [#106](https://github.com/cap-java/cds-ai/issues/106) + ### 8. Custom Schema Synchronisation _(S4)_ One possible enhancement is to let developers define custom document type extraction schemas in the CDS model via annotations, with the plugin syncing these to Document AI automatically at deploy time or startup - removing the need for manual configuration in the Document AI workspace. +**Tracking issue:** [#107](https://github.com/cap-java/cds-ai/issues/107) + ### 9. Customisable Extraction Templates Right now, every submission requires constructing the Document AI `options` JSON by hand. A template mechanism could let developers define named configurations - document type, schema ID, field selection, confidence thresholds - declaratively in the CDS model or `application.yaml`, and just reference the template name at submission time. +**Tracking issue:** [#116](https://github.com/cap-java/cds-ai/issues/116) + ### 10. Local Mock Mode _(S7)_ A mock mode returning configurable static extraction results without a real Document AI binding would make local development more convenient. -### 11. Application-Level Outbound Channels +**Tracking issue:** [#102](https://github.com/cap-java/cds-ai/issues/102) + +### 11. Configurable result delivery channels The plugin currently delivers results only via the `DocumentExtractionResult` CDS event. It could be worth exploring additional delivery channels so consuming applications can receive results through whatever channel fits their architecture. +**Tracking issue:** [#115](https://github.com/cap-java/cds-ai/issues/115) + ### 12. `cds add document-ai` Scaffold Command _(S9)_ A `cds add document-ai` CLI command could set up the Document AI service binding in `mta.yaml`, enable the persistent outbox in `application.yaml`, and generate boilerplate handler stubs - lowering the barrier significantly for new adopters. +**Tracking issue:** [#105](https://github.com/cap-java/cds-ai/issues/105) + ### 13. OData API Support _(S10)_ For applications on higher-tier plans, it could be worth exploring the Document AI OData API as an alternative transport, enabling richer querying and result navigation. +**Tracking issue:** [#99](https://github.com/cap-java/cds-ai/issues/99) + ### 14. Terminal Job Cleanup _(S11)_ A configurable retention policy that deletes or archives `ExtractionJob` rows after they've been in `DONE` or `FAILED` status for a set period would prevent unbounded table growth on high-volume deployments. +**Tracking issue:** [#103](https://github.com/cap-java/cds-ai/issues/103) + ### 15. Malware Scanning _(S8)_ It may be worth assessing whether documents should be scanned via SAP Malware Scanning Service before being forwarded to Document AI - particularly for multitenant deployments where uploaded content is less trusted. -### 16. Multiple Service Binding Support _(S12)_ +**Tracking issue:** [#104](https://github.com/cap-java/cds-ai/issues/104) -Supporting multiple Document AI bindings could allow applications to route documents to different instances based on context - document type, region, business unit. This would need a binding selection strategy, either convention-based or configurable via annotations or `application.yaml`. --- diff --git a/cds-starter-ai/README.md b/cds-starter-ai/README.md new file mode 100644 index 0000000..2483ebc --- /dev/null +++ b/cds-starter-ai/README.md @@ -0,0 +1,49 @@ +# cds-starter-ai + +A zero-source convenience module that bundles all SAP AI plugins for CAP Java into a single dependency. + +## What it includes + +| Module | Scope | Description | +|---|---|---| +| [`cds-feature-ai-core`](../cds-feature-ai-core/README.md) | `compile` | Bridges CAP Java to SAP AI Core — resource groups, deployments, configurations, inference client | +| [`cds-feature-recommendations`](../cds-feature-recommendations/README.md) | `runtime` | AI-powered field recommendations for Fiori Elements draft UIs | + +`cds-feature-sap-document-ai` is currently **not** included as the plugin is not yet fully multi-tenant-ready. It will be added to the starter once MT support is complete. Add it as an explicit dependency if you need it today. + +## Setup + +```xml + + com.sap.cds + cds-starter-ai + ${cds-ai.version} + +``` + +Both plugins auto-register via Java's `ServiceLoader` — no further code changes are required. + +For the Node.js side of `cds-feature-recommendations`, also add the `@cap-js/ai` CDS plugin to your `package.json`: + +```json +{ + "dependencies": { + "@cap-js/ai": "^1" + } +} +``` + +## Prerequisites + +- Java 21+ +- CAP Java 4.9+ +- An [SAP AI Core](https://help.sap.com/docs/sap-ai-core) service binding (for production use) +- A [SAP Document Information Extraction](https://help.sap.com/docs/document-ai) service binding (once `cds-feature-sap-document-ai` is included) + +Without the respective bindings the plugins fall back to mock implementations — useful for local development. + +## Related + +- [SAP AI Core Documentation](https://help.sap.com/docs/sap-ai-core) +- [SAP AI SDK for Java](https://github.com/SAP/ai-sdk-java) +- [`cds-feature-sap-document-ai`](../cds-feature-sap-document-ai/README.md) — standalone document extraction plugin diff --git a/integration-tests/README.md b/integration-tests/README.md index 42a660e..38e4622 100644 --- a/integration-tests/README.md +++ b/integration-tests/README.md @@ -45,7 +45,7 @@ Aggregated code coverage is produced by the `coverage-report/` module at the pro 1. Each module that runs tests has the JaCoCo agent attached (`prepare-agent`), which writes a `target/jacoco.exec` file during test execution. 2. The `coverage-report` module (built last in the reactor) merges all `.exec` files into a single `target/jacoco-merged.exec`. -3. It then generates an aggregated HTML/XML report via `jacoco:report-aggregate` and runs `jacoco:check` against configurable thresholds. +3. It then generates an aggregated HTML/XML report via `jacoco:report-aggregate`. Coverage thresholds are enforced by SonarQube in the pipeline, not by JaCoCo check goals. ### Generating the report @@ -61,12 +61,7 @@ coverage-report/target/site/jacoco-aggregate/index.html ### Thresholds -Per-module thresholds are defined in `coverage-report/pom.xml`: - -| Module | Instruction | Branch | Complexity | -|--------|-------------|--------|------------| -| `cds-feature-ai-core` | 0% | 0% | 0% | -| `cds-feature-recommendations` | 80% | 80% | 80% | +Coverage thresholds are enforced by SonarQube in the CI pipeline (80% on new code). ### Coverage data sources @@ -74,5 +69,6 @@ The merged report combines execution data from: - `cds-feature-ai-core/target/jacoco.exec` (unit tests) - `cds-feature-recommendations/target/jacoco.exec` (unit tests) +- `cds-feature-sap-document-ai/target/jacoco.exec` (unit tests) - `integration-tests/spring/target/jacoco.exec` (integration tests) - `integration-tests/mtx-local/srv/target/jacoco.exec` (MTX integration tests, only when profile is active) diff --git a/samples/bookshop/dummy invoice.pdf b/samples/bookshop/dummy invoice.pdf new file mode 100644 index 0000000..2309f5c Binary files /dev/null and b/samples/bookshop/dummy invoice.pdf differ