fix: improve custom SAST rule activation, filtering semantics + config observability

docs/github-action.md

@@ -670,8 +670,12 @@ jobs:
 
 ### Custom Rule Configuration
 
+Use custom rules from your repository by setting `use_custom_sast_rules` and
+`custom_sast_rule_path`. This path is resolved relative to `GITHUB_WORKSPACE`
+in GitHub Actions.
+
 ```yaml
-name: Security Scan with Custom Rules
+name: Security Scan with Custom SAST Rules
 on:
   pull_request:
     types: [opened, synchronize, reopened]
@@ -692,24 +696,31 @@ jobs:
           GITHUB_PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number }}
         with:
           github_token: ${{ secrets.GITHUB_TOKEN }}
-          
-          # Enable Python SAST
+
+          # Enable SAST languages you expect to run.
           python_sast_enabled: 'true'
-
-          # Enable specific Python rules
-          python_enabled_rules: 'sql-injection,xss,hardcoded-credentials'
-
-          # Disable noisy rules
-          python_disabled_rules: 'unused-import,line-too-long'
-
-          # JavaScript with custom rules
           javascript_sast_enabled: 'true'
+
+          # Enable custom rules from repository path.
+          use_custom_sast_rules: 'true'
+          custom_sast_rule_path: '.socket/rules'
+
+          # Optional: to avoid allowlist exclusions, run all rules for enabled languages.
+          all_rules_enabled: 'true'
+
+          # Optional: enable specific bundled or custom rule IDs.
           javascript_enabled_rules: 'eval-usage,prototype-pollution'
 
           # Ignore one or more SAST rules globally or for exact repo-relative files
           sast_ignore_overrides: 'js-sql-injection:index.js'
 ```
 
+Important behavior:
+- `socket_security_api_key` + `socket_org` enables dashboard config loading.
+- Dashboard/API settings override overlapping `with:` values.
+- `<language>_enabled_rules` is an allowlist and can suppress custom rule IDs.
+- `all_rules_enabled: 'true'` disables allowlist filtering for enabled languages.
+
 `sast_ignore_overrides` supports:
 - `rule_id` to ignore a SAST rule everywhere in the repo
 - `rule_id:path` to ignore a SAST rule for one exact repo-relative file
@@ -755,6 +766,8 @@ See [`action.yml`](../action.yml) for the complete list of inputs.
 **Rule Configuration (per language):**
 - `<language>_enabled_rules` — Comma-separated rules to enable
 - `<language>_disabled_rules` — Comma-separated rules to disable
+- `use_custom_sast_rules` — Enable custom SAST rule discovery from repo files
+- `custom_sast_rule_path` — Relative path to custom SAST rule directory
 - `sast_ignore_overrides` — Comma-separated `rule_id` or `rule_id:path` SAST ignore overrides
 
 **Security Scanning:**

docs/parameters.md

@@ -212,6 +212,10 @@ Use custom SAST rules instead of bundled rules (falls back to bundled rules for
 socket-basics --python --use-custom-sast-rules
 ```
 
+When this is enabled, custom rules are loaded from YAML files under
+`--custom-sast-rule-path`. Each rule must include a `languages` list so Socket
+Basics can map it to the correct OpenGrep language rule file.
+
 ### `--custom-sast-rule-path CUSTOM_SAST_RULE_PATH`
 Relative path to custom SAST rules directory (relative to workspace if set, otherwise cwd).
 
@@ -224,6 +228,11 @@ Relative path to custom SAST rules directory (relative to workspace if set, othe
 socket-basics --python --use-custom-sast-rules --custom-sast-rule-path "my_custom_rules"
 ```
 
+Custom rule file notes:
+- `.yml` and `.yaml` files are discovered recursively.
+- Files ending in `.test.yml` or `.test.yaml` are ignored.
+- Rules without `languages` are skipped.
+
 ### Language-Specific Rule Configuration
 
 For each language, you can enable or disable specific rules:
@@ -575,7 +584,9 @@ All notification integrations support environment variables as alternatives to C
 
 | Variable | Description |
 |----------|-------------|
-| `INPUT_OPENGREP_RULES_DIR` | Custom directory containing SAST rules |
+| `INPUT_OPENGREP_RULES_DIR` | Override directory for bundled OpenGrep rule files (`*.yml`) |
+| `INPUT_USE_CUSTOM_SAST_RULES` | Enable repository custom SAST rules |
+| `INPUT_CUSTOM_SAST_RULE_PATH` | Relative directory path for repository custom SAST rules |
 | `INPUT_SAST_IGNORE_OVERRIDES` | Comma-separated `rule_id` or `rule_id:path` SAST ignore overrides |
 
 ## Configuration File
@@ -593,6 +604,8 @@ You can provide configuration via a JSON file using `--config`:
 
   "python_sast_enabled": true,
   "javascript_sast_enabled": true,
+  "use_custom_sast_rules": true,
+  "custom_sast_rule_path": ".socket/rules",
   "go_sast_enabled": true,
   "sast_ignore_overrides": "js-sql-injection:index.js",
 
@@ -617,17 +630,18 @@ You can provide configuration via a JSON file using `--config`:
 Configuration is merged in the following order (later sources override earlier ones):
 
 1. Default values
-2. JSON configuration file (via `--config`)
-3. Environment variables
-4. Command-line arguments
+2. Environment variables
+3. Socket Basics API configuration (when available and no `--config` file is used)
+4. JSON configuration file (via `--config`)
+5. Command-line arguments
 
 **Example:**
 ```bash
-# JSON file sets python_sast_enabled: true
-# Environment has PYTHON_SAST_ENABLED=false
+# Environment sets python_sast_enabled=true
+# Dashboard/API sets python_sast_enabled=false
 # CLI has --javascript
-# Result: JavaScript enabled, Python disabled (env override), other settings from JSON
-socket-basics --config config.json --javascript
+# Result: JavaScript enabled, Python follows dashboard/API value, other settings from env/API
+socket-basics --javascript
 ```
 
 ## Common Usage Patterns

socket_basics/core/config.py

@@ -1124,6 +1124,10 @@ def normalize_api_config(api_config: Dict[str, Any]) -> Dict[str, Any]:
 
         # OpenGrep/SAST Configuration
         'openGrepNotificationMethod': 'opengrep_notification_method',
+        'useCustomSastRules': 'use_custom_sast_rules',
+        'customSastRulePath': 'custom_sast_rule_path',
+        # Accept common pluralized variant for robustness.
+        'customSastRulesPath': 'custom_sast_rule_path',
 
         # Socket Tier 1
         'socketTier1Enabled': 'socket_tier_1_enabled',
@@ -1231,13 +1235,15 @@ def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict
     Returns:
         Merged configuration dictionary
     """
+    logger = logging.getLogger(__name__)
+
     # Start with environment defaults (lowest priority)
     config = load_config_from_env()
+    logger.info("Configuration sources: environment defaults loaded")
 
     # Override with Socket Basics API config if no explicit JSON config provided
     # API config takes precedence over environment defaults
     if not json_config:
-        logger = logging.getLogger(__name__)
         logger.debug(" No JSON config provided, attempting to load Socket Basics API config")
         socket_basics_config = load_socket_basics_config()
         logger.debug(f" Socket Basics API config result: {socket_basics_config is not None}")
@@ -1254,7 +1260,10 @@ def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict
                     continue
                 filtered_config[k] = v
             config.update(filtered_config)
-            logging.getLogger(__name__).info("Loaded Socket Basics API configuration (overrides environment defaults)")
+            if bool(filtered_config.get('socket_has_enterprise', False)):
+                logging.getLogger(__name__).info("Loaded Socket Basics API configuration (overrides environment defaults)")
+            else:
+                logging.getLogger(__name__).info("Loaded Socket plan metadata (free/non-enterprise mode; no dashboard overrides)")
         else:
             logger.debug(" No Socket Basics API config loaded")
 
@@ -1276,6 +1285,13 @@ def merge_json_and_env_config(json_config: Dict[str, Any] | None = None) -> Dict
 
     # Note: CLI arguments are handled separately and take highest priority
     # They override the config object after this merge completes
+    logger.info(
+        "Effective custom SAST config: use_custom_sast_rules=%s custom_sast_rule_path=%s all_languages_enabled=%s all_rules_enabled=%s",
+        bool(config.get('use_custom_sast_rules', False)),
+        config.get('custom_sast_rule_path', ''),
+        bool(config.get('all_languages_enabled', False)),
+        bool(config.get('all_rules_enabled', False)),
+    )
 
     return config
 
@@ -1314,9 +1330,9 @@ def add_dynamic_cli_args(parser: argparse.ArgumentParser):
                 if param_type == 'bool':
                     parser.add_argument(option, action='store_true', help=description)
                 elif param_type == 'str':
-                    parser.add_argument(option, type=str, default=default, help=description)
+                    parser.add_argument(option, type=str, default=None, help=description)
                 elif param_type == 'int':
-                    parser.add_argument(option, type=int, default=default, help=description)
+                    parser.add_argument(option, type=int, default=None, help=description)
 
     except Exception as e:
         logging.getLogger(__name__).warning("Warning: Could not load dynamic CLI args: %s", e)
@@ -1346,9 +1362,9 @@ def add_dynamic_cli_args(parser: argparse.ArgumentParser):
                     if p_type == 'bool':
                         parser.add_argument(option, action='store_true', help=desc)
                     elif p_type == 'int':
-                        parser.add_argument(option, type=int, default=default, help=desc)
+                        parser.add_argument(option, type=int, default=None, help=desc)
                     else:
-                        parser.add_argument(option, type=str, default=default, help=desc)
+                        parser.add_argument(option, type=str, default=None, help=desc)
     except Exception:
         pass
 
@@ -1357,7 +1373,7 @@ def parse_cli_args():
     """Parse command line arguments and return argument parser"""
     parser = argparse.ArgumentParser(description='Socket Security Basics - Dynamic security scanning')
     parser.add_argument('--config', type=str, 
-                       help='Path to JSON configuration file. JSON config is merged with environment variables (environment takes precedence)')
+                       help='Path to JSON configuration file. JSON config is merged with environment variables (JSON takes precedence)')
     parser.add_argument('--output', type=str, default='.socket.facts.json', 
                        help='Output file name (default: .socket.facts.json)')
     parser.add_argument('--workspace', type=str, help='Workspace directory to scan')

socket_basics/core/connector/opengrep/__init__.py

@@ -40,6 +40,12 @@ def scan(self) -> Dict[str, Any]:
 			rule_files = self.config.build_opengrep_rules() or []
 		except Exception:
 			rule_files = []
+		logger.info(
+			"OpenGrep config summary: all_languages_enabled=%s all_rules_enabled=%s requested_rule_files=%s",
+			bool(self.config.get('all_languages_enabled', False)),
+			bool(self.config.get('all_rules_enabled', False)),
+			rule_files,
+		)
 
 		# If no languages selected and not explicitly allowing all, skip
 		if not rule_files and not self.config.get('all_languages_enabled', False):
@@ -55,9 +61,35 @@ def scan(self) -> Dict[str, Any]:
 			logger.info('No scan targets to analyze (scoped scan matched no existing files); skipping OpenGrep')
 			return {}
 
+		# Locate bundled rules directory for fallback and all-language expansion.
+		module_dir = Path(__file__).resolve().parents[3]
+		bundled_rules_dir = module_dir / 'rules'
+		rules_dir = self.config.get('opengrep_rules_dir') or (str(bundled_rules_dir) if bundled_rules_dir.exists() else None)
+		if not rules_dir:
+			logger.error('No rules directory found')
+			return {}
+
+		if not rule_files and self.config.get('all_languages_enabled', False):
+			try:
+				rule_files = [
+					p.name
+					for p in Path(rules_dir).glob('*.yml')
+					if p.name != 'tests.yml'
+				]
+				logger.info("Expanded all-languages scan to rule files: %s", rule_files)
+			except Exception:
+				logger.debug('Failed expanding all-languages into rule files', exc_info=True)
+				rule_files = []
+
 		# Check if custom rules mode is enabled
 		custom_rules_path = self.config.get_custom_rules_path()
 		custom_rule_files: Dict[str, Path] = {}
+		logger.info(
+			"Custom SAST requested=%s custom_path=%s resolved_path=%s",
+			bool(self.config.get('use_custom_sast_rules', False)),
+			self.config.get('custom_sast_rule_path', ''),
+			str(custom_rules_path) if custom_rules_path else '(none)',
+		)
 
 		if custom_rules_path:
 			logger.info(f"Custom SAST rules enabled, loading from: {custom_rules_path}")
@@ -68,19 +100,16 @@ def scan(self) -> Dict[str, Any]:
 				logger.error(f"Failed to build custom rule files: {e}", exc_info=True)
 				custom_rule_files = {}
 
-		# Locate bundled rules directory for fallback
-		module_dir = Path(__file__).resolve().parents[3]
-		bundled_rules_dir = module_dir / 'rules'
-		rules_dir = self.config.get('opengrep_rules_dir') or (str(bundled_rules_dir) if bundled_rules_dir.exists() else None)
-		if not rules_dir:
-			logger.error('No rules directory found')
-			return {}
-
 		# Read filtered rule definitions if available
 		try:
 			filtered = self.config.build_filtered_opengrep_rules() or {}
 		except Exception:
 			filtered = {}
+		if filtered:
+			filtered_counts = {k: len(v or []) for k, v in filtered.items()}
+			logger.info("Per-language enabled-rule filters detected: %s", filtered_counts)
+		else:
+			logger.info("Per-language enabled-rule filters disabled for this run")
 
 		# Debugging: log computed rule files and filtered rules for diagnosis
 		try:
@@ -98,25 +127,42 @@ def scan(self) -> Dict[str, Any]:
 		# Process all enabled languages - use filtered rules if specified, otherwise use all rules
 		for rf in rule_files:
 			# Check if we have a custom rule file for this language
+			using_custom_rules = bool(custom_rule_files and rf in custom_rule_files)
 			if custom_rule_files and rf in custom_rule_files:
 				p = custom_rule_files[rf]
-				logger.info(f"Using custom rules for {rf}")
+				logger.info("Using custom rules for %s from %s", rf, p)
 			else:
 				# Fall back to bundled rules
 				p = Path(rules_dir) / rf
 				if not p.exists():
 					logger.debug('Rule file missing: %s', p)
 					continue
+				logger.info("Using bundled rules for %s from %s", rf, p)
 
 			# Check if this language has specific rules enabled (filtered mode)
 			if filtered and rf in filtered:
 				enabled_ids = filtered[rf]
-				logger.debug(f"Using filtered rules for {rf}: {len(enabled_ids)} rules enabled")
+				logger.info("Filtering rules for %s: %d enabled IDs configured", rf, len(enabled_ids))
 				try:
 					with open(p, 'r') as fh:
 						data = yaml.safe_load(fh) or {}
 					all_ids = [r.get('id') for r in (data.get('rules') or []) if r.get('id')]
-					to_exclude = [rid for rid in all_ids if rid not in (enabled_ids or [])]
+					# Custom-rule mode can coexist with legacy bundled allowlists.
+					# If none of the configured enabled IDs match custom IDs, keep all
+					# custom IDs active to avoid silently disabling user-authored rules.
+					if using_custom_rules:
+						matched_enabled_ids = [rid for rid in all_ids if rid in (enabled_ids or [])]
+						if enabled_ids and not matched_enabled_ids:
+							logger.warning(
+								"No configured enabled-rule IDs matched custom rules for %s; using all custom rules from %s",
+								rf,
+								p,
+							)
+							config_args.extend(['--config', str(p)])
+							continue
+						to_exclude = [rid for rid in all_ids if rid not in matched_enabled_ids]
+					else:
+						to_exclude = [rid for rid in all_ids if rid not in (enabled_ids or [])]
 					config_args.extend(['--config', str(p)])
 					for ex in to_exclude:
 						config_args.extend(['--exclude-rule', ex])
@@ -754,4 +800,3 @@ def generate_notifications(self, components: List[Dict[str, Any]]) -> Dict[str,
 		notifications_by_notifier['webhook'] = webhook.format_notifications(groups)
 
 		return notifications_by_notifier
-