Skip to content

Commit aba0d9a

Browse files
authored
Merge pull request #44002 from github/repo-sync
Repo sync
2 parents a58ccf5 + 4c556ca commit aba0d9a

10 files changed

Lines changed: 173 additions & 24 deletions

File tree

Lines changed: 149 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,149 @@
1+
---
2+
title: Assessing the impact of GitHub Secret Protection
3+
intro: 'Measure how {% data variables.product.prodname_GH_secret_protection_always %} reduces secret exposure across your organization, so you can demonstrate value and identify areas to strengthen your security posture.'
4+
allowTitleToDifferFromFilename: true
5+
shortTitle: Assess GHSP impact
6+
versions:
7+
fpt: '*'
8+
ghec: '*'
9+
ghes: '*'
10+
contentType: tutorials
11+
category:
12+
- Protect your secrets
13+
---
14+
15+
## Introduction
16+
17+
After enabling {% data variables.product.prodname_GH_secret_protection_always %} (GHSP) for your organization, you'll want to assess its impact and understand how it's protecting your organization. This tutorial walks you through accessing secret-related data and interpreting the results to measure GHSP performance.
18+
19+
In this tutorial, you'll learn how to:
20+
* Access your organization's security overview to view {% data variables.product.prodname_secret_scanning %} data
21+
* Review the {% data variables.product.prodname_secret_risk_assessment %} (SRA) report
22+
* Compare and analyze the data to assess GHSP's impact
23+
24+
If you don't have a historic SRA report from before your GHSP rollout, you can still assess GHSP's effectiveness. Skip ahead to [Step 4: Analyze security overview data trends](#step-4-analyze-security-overview-data-trends).
25+
26+
## Prerequisites
27+
28+
* You need to have the organization owner or security manager role.
29+
* {% data variables.product.prodname_secret_protection %} must be enabled for your organization.
30+
31+
## Step 1: Access the organization-level security overview
32+
33+
The security overview provides real-time data about {% data variables.secret-scanning.alerts %} across your organization.
34+
35+
{% data reusables.organizations.navigate-to-org %}
36+
{% data reusables.organizations.security-overview %}
37+
1. On the security overview page, click the **Risk** tab to view secret scanning data.
38+
The overview shows:
39+
* Total number of open {% data variables.secret-scanning.alerts %}
40+
* Alert trends over time
41+
* Breakdown by repository
42+
* Alert severity distribution
43+
44+
## Step 2: View your {% data variables.product.prodname_secret_risk_assessment %} report
45+
46+
If you previously ran a SRA report, you can access the report to establish a baseline.
47+
48+
{% data reusables.organizations.navigate-to-org %}
49+
{% data reusables.organizations.security-overview %}
50+
{% data reusables.security-overview.open-assessments-view %}
51+
1. Review the key metrics from the assessment, including:
52+
* Number of exposed secrets detected
53+
* Types of secrets found
54+
* Repositories with the highest risk
55+
* Recommended remediation actions
56+
57+
> [!NOTE] The SRA report represents a point-in-time snapshot of your secret exposure before or during your GHSP implementation.
58+
59+
## Step 3: Compare SRA data with current security overview
60+
61+
The SRA report is a **point-in-time** snapshot taken before or during your GHSP rollout, while the security overview shows **real-time** data that updates as alerts are opened and resolved. To make a meaningful comparison, you need to ensure both datasets cover the same secret types.
62+
63+
### Filter to comparable pattern types
64+
65+
The SRA report only detects **provider patterns** and **generic patterns**. The security overview, however, may also include results from custom patterns you've configured since enabling GHSP. To ensure an accurate comparison, filter the security overview to the same pattern types the SRA covers.
66+
67+
#### Using the UI
68+
69+
In the security overview **Risk** tab, use the filter bar to narrow results to provider and generic patterns only, excluding any custom patterns.
70+
71+
#### Using the API
72+
73+
Alternatively, you can use the REST API to programmatically retrieve alerts filtered by secret type. For example, to list only default (provider) {% data variables.secret-scanning.alerts %} for a repository:
74+
75+
```shell copy
76+
gh api \
77+
-H "Accept: application/vnd.github+json" \
78+
/orgs/ORG/secret-scanning/alerts --paginate
79+
```
80+
81+
This returns alerts for default patterns only. To also include generic patterns in your results, pass the specific token names using the `secret_type` parameter.
82+
83+
For more information, see [AUTOTITLE](/rest/secret-scanning/secret-scanning).
84+
85+
### Build your comparison
86+
87+
1. Using the filtered data, create a comparison table with these key metrics:
88+
89+
| Metric | SRA report (Baseline) | Current security overview (Filtered) | Change |
90+
|--------|----------------------|--------------------------------------|--------|
91+
| Total exposed secrets | [SRA number] | [Current number] | [Difference] |
92+
| Critical alerts | [SRA number] | [Current number] | [Difference] |
93+
| Affected repositories | [SRA number] | [Current number] | [Difference] |
94+
95+
1. Calculate the percentage change for each metric:
96+
* **Positive impact indicators:** Reduction in total exposed secrets, fewer critical alerts
97+
* **Areas for improvement:** New alerts appearing, specific repositories with increasing trends
98+
99+
1. Note any significant differences in:
100+
* Secret types being detected
101+
* Repository coverage
102+
* Alert resolution rates
103+
104+
## Step 4: Analyze security overview data trends
105+
106+
Even without an SRA report, you can assess GHSP effectiveness by analyzing trends in the security overview.
107+
108+
{% data reusables.organizations.navigate-to-org %}
109+
{% data reusables.organizations.security-overview %}
110+
1. In the security overview **Risk** tab, look at the trend graph showing {% data variables.secret-scanning.alerts %} over time.
111+
1. Identify patterns:
112+
* **Declining trend:** Indicates successful remediation and prevention
113+
* **Plateau:** May suggest steady state or need for increased awareness
114+
* **Rising trend:** May indicate increased detection coverage or new secret introduction
115+
116+
1. Click on individual repositories to drill down into specific alert details.
117+
1. Review the alert resolution rate:
118+
* Navigate to the **{% data variables.product.prodname_security_and_quality_tab %}** tab for your organization.
119+
* Under "Findings", Click **{% data variables.product.prodname_secret_scanning_caps %}**.
120+
* Check how many alerts have been closed versus the number of alerts that remain open.
121+
* Select the alert type you're interested in.
122+
* Assess average time to resolution.
123+
124+
## Step 5: Interpret the results and take action
125+
126+
Based on your analysis, determine the next steps.
127+
128+
### If you're seeing positive trends
129+
130+
* Document the improvement to demonstrate GHSP value
131+
* Identify successful practices to replicate across other repositories
132+
* Consider expanding GHSP coverage to additional repositories or organizations
133+
134+
### If you're seeing areas for improvement
135+
136+
* Review repositories with increasing alerts or slow resolution times
137+
* Provide additional training to development teams
138+
* Assess whether custom patterns need to be configured
139+
* Check if push protection is enabled to prevent new secrets from being introduced
140+
141+
### Ongoing monitoring
142+
143+
* Schedule regular reviews (weekly or monthly) of the security overview
144+
* Set up notifications for new {% data variables.secret-scanning.alerts %}
145+
* Track metrics over time to demonstrate continuous improvement
146+
147+
## Further reading
148+
149+
* To understand {% data variables.product.prodname_secret_scanning %} metrics in detail, see [AUTOTITLE](/code-security/security-overview/viewing-security-insights).

content/code-security/tutorials/remediate-leaked-secrets/index.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ versions:
88
contentType: tutorials
99
children:
1010
- /calculating-the-cost-savings-of-push-protection
11+
- /assessing-ghsp-impact
1112
- /evaluating-alerts
1213
- /remediating-a-leaked-secret
1314
---
14-

content/copilot/concepts/agents/about-agent-skills.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@ You can also use `gh skill` in {% data variables.product.prodname_cli %} to disc
2525
{% data variables.product.prodname_copilot_short %} supports:
2626

2727
* Project skills, stored in your repository (`.github/skills`, `.claude/skills`, or `.agents/skills`)
28-
* Personal skills, stored in your home directory and shared across projects (`~/.copilot/skills`, `~/.claude/skills`, or `~/.agents/skills`)
28+
* Personal skills, stored in your home directory and shared across projects (`~/.copilot/skills` or `~/.agents/skills`)
2929

3030
Support for organization-level and enterprise-level skills is coming soon.
3131

content/copilot/concepts/billing/copilot-requests.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ The following {% data variables.product.prodname_copilot_short %} features can u
4242
| [{% data variables.copilot.copilot_chat_short %}](/copilot/using-github-copilot/copilot-chat) | {% data variables.copilot.copilot_chat_short %} uses **one premium request** per user prompt, multiplied by the model's rate. This includes ask, edit, agent, and plan modes in {% data variables.copilot.copilot_chat_short %} in an IDE. | {% data variables.product.prodname_copilot_short %} premium requests |
4343
| [{% data variables.copilot.copilot_cli_short %}](/copilot/concepts/agents/about-copilot-cli) | Each prompt to {% data variables.copilot.copilot_cli_short %} uses **one premium request** with the default model. For other models, this is multiplied by the model's rate. | {% data variables.product.prodname_copilot_short %} premium requests |
4444
| [{% data variables.product.prodname_copilot_short %} code review](/copilot/using-github-copilot/code-review/using-copilot-code-review) | Each time {% data variables.product.prodname_copilot_short %} reviews a pull request (when assigned as a reviewer) or reviews code in your IDE, **one premium request** is consumed. | {% data variables.product.prodname_copilot_short %} premium requests |
45-
| [{% data variables.copilot.copilot_cloud_agent %}](/copilot/concepts/about-copilot-cloud-agent) | {% data variables.copilot.copilot_cloud_agent %} uses **one premium request** per session, multiplied by the model's rate. A session begins when you prompt {% data variables.product.prodname_copilot_short %} to undertake a task. In addition, each real-time steering comment made during an active session uses **one premium request** per session, multiplied by the model's rate. | {% data variables.copilot.copilot_cloud_agent %} premium requests |
45+
| [{% data variables.copilot.copilot_cloud_agent %}](/copilot/concepts/agents/cloud-agent/about-cloud-agent) | {% data variables.copilot.copilot_cloud_agent %} uses **one premium request** per session, multiplied by the model's rate. A session begins when you prompt {% data variables.product.prodname_copilot_short %} to undertake a task. In addition, each real-time steering comment made during an active session uses **one premium request** per session, multiplied by the model's rate. | {% data variables.copilot.copilot_cloud_agent %} premium requests |
4646
| [{% data variables.copilot.copilot_spaces %}](/copilot/using-github-copilot/copilot-spaces/about-organizing-and-sharing-context-with-copilot-spaces) | {% data variables.copilot.copilot_spaces %} uses **one premium request** per user prompt, multiplied by the model's rate. | {% data variables.product.prodname_copilot_short %} premium requests |
4747
| [{% data variables.product.prodname_spark_short %}](/copilot/tutorials/building-ai-app-prototypes) | Each prompt to {% data variables.product.prodname_spark_short %} uses a fixed rate of **four premium requests**. | {% data variables.product.prodname_spark_short %} premium requests |
4848
| [{% data variables.product.prodname_openai_codex %} {% data variables.product.prodname_vscode %} integration](/copilot/concepts/agents/openai-codex) | While in preview, each prompt to {% data variables.product.prodname_openai_codex %} uses **one premium request** multiplied by the model multiplier rates. | {% data variables.product.prodname_copilot_short %} premium requests |

content/copilot/reference/copilot-cli-reference/cli-command-reference.md

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -141,7 +141,7 @@ COPILOT_GITHUB_TOKEN=github_pat_... copilot
141141
| `/plan [PROMPT]` | Create an implementation plan before coding. |
142142
| `/plugin [marketplace\|install\|uninstall\|update\|list] [ARGS...]` | Manage plugins and plugin marketplaces. See [AUTOTITLE](/copilot/concepts/agents/copilot-cli/about-cli-plugins). |
143143
| `/pr [view\|create\|fix\|auto]` | Manage pull requests for the current branch. See [AUTOTITLE](/copilot/how-tos/copilot-cli/manage-pull-requests). |
144-
| `/remote` | Enable remote access to this session from {% data variables.product.prodname_dotcom_the_website %} and {% data variables.product.prodname_mobile %}. See [AUTOTITLE](/copilot/how-tos/copilot-cli/steer-remotely). |
144+
| `/remote [on\|off]` | Show remote status (if no argument provided), enable remote steering (`on`), or end the remote connection (`off`). See [AUTOTITLE](/copilot/how-tos/copilot-cli/steer-remotely). |
145145
| `/rename [NAME]` | Rename the current session (auto-generates a name if omitted; alias for `/session rename`). |
146146
| `/research TOPIC` | Run a deep research investigation using {% data variables.product.github %} search and web sources. See [AUTOTITLE](/copilot/concepts/agents/copilot-cli/research). |
147147
| `/reset-allowed-tools` | Reset the list of allowed tools. |
@@ -397,7 +397,7 @@ Command hooks run shell scripts and are supported on all hook types.
397397

398398
#### Prompt hooks
399399

400-
Prompt hooks auto-submit text as if the user typed it. They are only supported on `sessionStart` and run before any initial prompt passed via `--prompt`. The text can be a natural language prompt or a slash command.
400+
Prompt hooks auto-submit text as if the user typed it. They are only supported on `sessionStart` and only fire for **new interactive sessions**. They do not fire on resume, and they do not fire in non-interactive prompt mode (`-p`). The text can be a natural language prompt or a slash command.
401401

402402
```json
403403
{
@@ -498,7 +498,7 @@ For `preToolUse` and `permissionRequest`, an HTTP hook failure is fail-open: the
498498
| `subagentStop` | A subagent completes. | Yes — can block and force continuation. |
499499
| `subagentStart` | A subagent is spawned (before it runs). Returns `additionalContext` prepended to the subagent's prompt. Supports `matcher` to filter by agent name. | No — cannot block creation. |
500500
| `preCompact` | Context compaction is about to begin (manual or automatic). Supports `matcher` to filter by trigger (`"manual"` or `"auto"`). | No — notification only. |
501-
| `permissionRequest` | Before showing a permission dialog to the user, after rule-based checks find no matching allow or deny rule. Supports `matcher` regex on `toolName`. | Yes — can allow or deny programmatically. |
501+
| `permissionRequest` | Fires before the permission service runs (rules engine, session approvals, auto-allow/auto-deny, and user prompting). If the merged hook output returns `behavior: "allow"` or `"deny"`, that decision short-circuits the normal permission flow. Supports `matcher` regex on `toolName`. | Yes — can allow or deny programmatically. |
502502
| `errorOccurred` | An error occurs during execution. | No |
503503
| `notification` | Fires asynchronously when the CLI emits a system notification (shell completion, agent completion or idle, permission prompts, elicitation dialogs). Fire-and-forget: never blocks the session. Supports `matcher` regex on `notification_type`. | Optional — can inject `additionalContext` into the session. |
504504

@@ -842,9 +842,11 @@ The `preToolUse` hook can control tool execution by writing a JSON object to std
842842

843843
### `permissionRequest` decision control
844844

845-
The `permissionRequest` hook fires when a tool-level permission dialog is about to be shown. It fires after rule-based permission checks find no matching allow or deny rule. Use it to approve or deny tool calls programmatically—especially useful in pipe mode (`-p`) and CI environments where no interactive prompt is available.
845+
The `permissionRequest` hook fires before the permission service runs—before rule checks, session approvals, auto-allow/auto-deny, and user prompting. If hooks return `behavior: "allow"` or `"deny"`, that decision short-circuits the normal permission flow. Returning nothing falls through to normal permission handling. Use it to approve or deny tool calls programmatically—especially useful in pipe mode (`-p`) and CI environments where no interactive prompt is available.
846846

847-
**Matcher:** Optional regex tested against `toolName`. When set, the hook fires only for matching tool names.
847+
All configured `permissionRequest` hooks run for each request (except `read` and `hook` permission kinds, which short-circuit before hooks). Hook outputs are merged with later hook outputs overriding earlier ones.
848+
849+
**Matcher:** Optional regex tested against `toolName`. Anchored as `^(?:pattern)$`; must match the full tool name. When set, the hook fires only for matching tool names.
848850

849851
Output JSON to stdout to control the permission decision:
850852

@@ -854,7 +856,7 @@ Output JSON to stdout to control the permission decision:
854856
| `message` | string | Reason fed back to the LLM when denying. |
855857
| `interrupt` | boolean | When `true` combined with `"deny"`, stops the agent entirely. |
856858

857-
Return empty output or `{}` to fall through to the default behavior (show the user dialog, or deny in pipe mode). For command hooks, exit code `2` is treated as a deny; stdout JSON (if any) is merged with `{"behavior":"deny"}`, and stderr is ignored.
859+
Return empty output or `{}` to fall through to the normal permission flow. For command hooks, exit code `2` is treated as a deny; stdout JSON (if any) is merged with `{"behavior":"deny"}`, and stderr is ignored.
858860

859861
### `notification` hook
860862

@@ -910,6 +912,7 @@ If `additionalContext` is returned, the text is injected into the session as a p
910912
| `grep` | Search file contents. |
911913
| `web_fetch` | Fetch web pages. |
912914
| `task` | Run subagent tasks. |
915+
| `ask_user` | Ask the user a clarifying question. |
913916

914917
If multiple hooks of the same type are configured, they execute in order. For `preToolUse`, if any hook returns `"deny"`, the tool is blocked. Exit codes apply to command hooks only—for HTTP hooks, see the [HTTP hook failure semantics](#http-hook-failure-semantics). For `postToolUseFailure` command hooks, exiting with code `2` causes stderr to be returned as recovery guidance for the assistant. For `permissionRequest` command hooks, exit code `2` is treated as a deny; stdout JSON (if any) is merged with `{"behavior":"deny"}`, and stderr is ignored. Hook failures (non-zero exit codes or timeouts) are logged and skipped—they never block agent execution.
915918

@@ -1091,7 +1094,6 @@ Skills are loaded from these locations in priority order (first found wins for d
10911094
| Parent `.github/skills/` | Inherited | Monorepo parent directory support. |
10921095
| `~/.copilot/skills/` | Personal | Personal skills for all projects. |
10931096
| `~/.agents/skills/` | Personal | Agent skills shared across all projects. |
1094-
| `~/.claude/skills/` | Personal | Claude-compatible personal location. |
10951097
| Plugin directories | Plugin | Skills from installed plugins. |
10961098
| `COPILOT_SKILLS_DIRS` | Custom | Additional directories (comma-separated). |
10971099
| (bundled with CLI) | Built-in | Skills shipped with the CLI. Lowest priority—overridable by any other source. |
@@ -1132,7 +1134,7 @@ Custom agents are specialized AI agents defined in Markdown files. The filename
11321134
| Scope | Location |
11331135
|-------|----------|
11341136
| Project | `.github/agents/` or `.claude/agents/` |
1135-
| User | `~/.copilot/agents/` or `~/.claude/agents/` |
1137+
| User | `~/.copilot/agents/` |
11361138
| Plugin | `<plugin>/agents/` |
11371139

11381140
Project-level agents take precedence over user-level agents. Plugin agents have the lowest priority.

content/copilot/reference/copilot-cli-reference/cli-config-dir-reference.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -94,7 +94,7 @@ For more information, see [AUTOTITLE](/copilot/how-tos/copilot-cli/customize-cop
9494

9595
### `hooks/`
9696

97-
Store user-level hook scripts here. These hooks apply to all your sessions. You can also define hooks inline in your user configuration file (`~/.copilot/config.json`) using the `hooks` key. Repository-level hooks (in `.github/hooks/`) are loaded alongside user-level hooks.
97+
Store user-level hook scripts here. These hooks apply to all your sessions. You can also define hooks inline in your user configuration file (`~/.copilot/settings.json`) using the `hooks` key. Repository-level hooks (in `.github/hooks/`) are loaded alongside user-level hooks.
9898

9999
For more information, see [AUTOTITLE](/copilot/how-tos/copilot-cli/customize-copilot/use-hooks).
100100

0 commit comments

Comments
 (0)