Skip to content

Commit d51177b

Browse files
AbacnAbacn
authored
Add a check to make sure GitHub action workflows are listed in readme (#37883)
1 parent ae6d624 commit d51177b

5 files changed

Lines changed: 32 additions & 3 deletions

File tree

‎.github/build.gradle‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,7 @@ buildscript {
2828
/** check that yml are valid */
2929
task check {
3030
doLast {
31+
def workflowDoc = new File("${project.projectDir}/workflows/README.md").text
3132
List<String> errors = []
3233
fileTree("${project.projectDir}/workflows").matching {
3334
include "*.yml"
@@ -68,6 +69,11 @@ task check {
6869
}
6970
}
7071
}
72+
73+
// Make sure the workflow is listed in README.md
74+
if ( fname.startsWith("beam_") && !workflowDoc.contains(fname)) {
75+
errors.add("Workflow ${fname} not listed in workflows/README.md");
76+
}
7177
}
7278
if (!errors.isEmpty()) {
7379
throw new GradleException("Check failed: " + errors.join('\n'))

‎.github/workflows/README.md‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -297,6 +297,13 @@ PreCommit Jobs run in a schedule and also get triggered in a PR if relevant sour
297297
| [ PreCommit Whitespace ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Whitespace.yml) | N/A |`Run Whitespace PreCommit`| [![.github/workflows/beam_PreCommit_Whitespace.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Whitespace.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Whitespace.yml?query=event%3Aschedule) |
298298
| [ PreCommit Xlang Generated Transforms ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Xlang_Generated_Transforms.yml) | N/A |`Run Xlang_Generated_Transforms PreCommit`| [![.github/workflows/beam_PreCommit_Xlang_Generated_Transforms.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Xlang_Generated_Transforms.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Xlang_Generated_Transforms.yml?query=event%3Aschedule) |
299299
| [ PreCommit YAML Xlang Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Yaml_Xlang_Direct.yml) | N/A |`Run Yaml_Xlang_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Yaml_Xlang_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Yaml_Xlang_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Yaml_Xlang_Direct.yml?query=event%3Aschedule) |
300+
| [ PreCommit Flink Container ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Flink_Container.yml) | N/A |`Run Flink Container PreCommit`| [![.github/workflows/beam_PreCommit_Flink_Container.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Flink_Container.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Flink_Container.yml?query=event%3Aschedule) |
301+
| [ PreCommit GoPrism ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPrism.yml) | N/A |`Run GoPrism PreCommit`| [![.github/workflows/beam_PreCommit_GoPrism.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPrism.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_GoPrism.yml?query=event%3Aschedule) |
302+
| [ PreCommit Integration and Load Test Framework ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_ItFramework.yml) | N/A |`Run It_Framework PreCommit`| [![.github/workflows/beam_PreCommit_ItFramework.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_ItFramework.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_ItFramework.yml?query=event%3Aschedule) |
303+
| [ PreCommit Java PVR Prism Loopback ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_PVR_Prism_Loopback.yml) | N/A |`Run Java_PVR_Prism_Loopback PreCommit`| [![.github/workflows/beam_PreCommit_Java_PVR_Prism_Loopback.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_PVR_Prism_Loopback.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_PVR_Prism_Loopback.yml?query=event%3Aschedule) |
304+
| [ PreCommit Java Solace IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Solace_IO_Direct.yml) | N/A |`Run Java_Solace_IO_Direct PreCommit`| [![.github/workflows/beam_PreCommit_Java_Solace_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Solace_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Java_Solace_IO_Direct.yml?query=event%3Aschedule) |
305+
| [ PreCommit Prism Python ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Prism_Python.yml) | ['3.10', '3.13'] |`Run Prism_Python PreCommit`| [![.github/workflows/beam_PreCommit_Prism_Python.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Prism_Python.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Prism_Python.yml?query=event%3Aschedule) |
306+
| [ PreCommit Python Dill tests with dill deps installed ](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Dill.yml) | ['3.12'] |`Run Python_Dill PreCommit`| [![.github/workflows/beam_PreCommit_Python_Dill.yml](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Dill.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PreCommit_Python_Dill.yml?query=event%3Aschedule) |
300307

301308
Additional PreCommit jobs running basic SDK unit test on a matrices of operating systems. These workflows were setup differently and currently do not support trigger phrases
302309

@@ -405,6 +412,10 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
405412
| [ PostCommit XVR Spark3 ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_XVR_Spark3.yml) | N/A |`beam_PostCommit_XVR_Spark3.json`| [![.github/workflows/beam_PostCommit_XVR_Spark3.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_XVR_Spark3.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_XVR_Spark3.yml?query=event%3Aschedule) |
406413
| [ PostCommit YAML Xlang Direct ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml) | N/A |`beam_PostCommit_Yaml_Xlang_Direct.json`| [![.github/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Yaml_Xlang_Direct.yml?query=event%3Aschedule) |
407414
| [ Python Validates Container Dataflow ARM ](https://github.com/apache/beam/actions/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml) | ['3.9','3.10','3.11','3.12'] |`beam_Python_ValidatesContainer_Dataflow_ARM.json`|[![.github/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml](https://github.com/apache/beam/actions/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Python_ValidatesContainer_Dataflow_ARM.yml?query=event%3Aschedule) |
415+
| [ PostCommit Java ValidatesRunner Dataflow Streaming TagEncodingV2 ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_TagEncodingV2.yml) | N/A |`beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_TagEncodingV2.json`| [![.github/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_TagEncodingV2.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_TagEncodingV2.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_TagEncodingV2.yml?query=event%3Aschedule) |
416+
| [ PostCommit Java ValidatesRunner Dataflow Streaming Engine ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_Engine.yml) | N/A |`beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_Engine.json`| [![.github/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_Engine.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_Engine.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming_Engine.yml?query=event%3Aschedule) |
417+
| [ PostCommit Python Portable Flink ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Portable_Flink.yml) | N/A |`beam_PostCommit_Python_Portable_Flink.json`| [![.github/workflows/beam_PostCommit_Python_Portable_Flink.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Portable_Flink.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Portable_Flink.yml?query=event%3Aschedule) |
418+
| [ PostCommit Python Xlang IO Direct ](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_IO_Direct.yml) | N/A |`beam_PostCommit_Python_Xlang_IO_Direct.json`| [![.github/workflows/beam_PostCommit_Python_Xlang_IO_Direct.yml](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_IO_Direct.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PostCommit_Python_Xlang_IO_Direct.yml?query=event%3Aschedule) |
408419

409420
### PerformanceTests and Benchmark Jobs
410421

@@ -446,6 +457,7 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
446457
| [ PerformanceTests XmlIOIT HDFS ](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml) | N/A | [![.github/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT_HDFS.yml?query=event%3Aschedule)
447458
| [ PerformanceTests XmlIOIT ](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT.yml) | N/A | [![.github/workflows/beam_PerformanceTests_XmlIOIT.yml](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_XmlIOIT.yml?query=event%3Aschedule)
448459
| [ PerformanceTests xlang KafkaIO Python ](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml) | N/A | [![.github/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_PerformanceTests_xlang_KafkaIO_Python.yml?query=event%3Aschedule)
460+
| [ Python Cost Benchmarks Dataflow ](https://github.com/apache/beam/actions/workflows/beam_Python_CostBenchmarks_Dataflow.yml) | N/A | [![.github/workflows/beam_Python_CostBenchmarks_Dataflow.yml](https://github.com/apache/beam/actions/workflows/beam_Python_CostBenchmarks_Dataflow.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Python_CostBenchmarks_Dataflow.yml?query=event%3Aschedule) |
449461

450462
### LoadTests Jobs
451463

@@ -502,6 +514,12 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
502514
| [ LoadTests Python ParDo Flink Streaming ](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_ParDo_Flink_Streaming.yml) | N/A | [![.github/workflows/beam_LoadTests_Python_ParDo_Flink_Streaming.yml](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_ParDo_Flink_Streaming.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_ParDo_Flink_Streaming.yml?query=event%3Aschedule)
503515
| [ LoadTests Python SideInput Dataflow Batch ](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_SideInput_Dataflow_Batch.yml) | N/A | [![.github/workflows/beam_LoadTests_Python_SideInput_Dataflow_Batch.yml](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_SideInput_Dataflow_Batch.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_SideInput_Dataflow_Batch.yml?query=event%3Aschedule)
504516
| [ LoadTests Python Smoke ](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_Smoke.yml) | N/A | [![.github/workflows/beam_LoadTests_Python_Smoke.yml](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_Smoke.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Python_Smoke.yml?query=event%3Aschedule)
517+
| [ LoadTests Java PubsubIO ](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Java_PubsubIO.yml) | N/A | [![.github/workflows/beam_LoadTests_Java_PubsubIO.yml](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Java_PubsubIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_LoadTests_Java_PubsubIO.yml?query=event%3Aschedule) |
518+
| [ StressTests Java BigQueryIO ](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigQueryIO.yml) | N/A | [![.github/workflows/beam_StressTests_Java_BigQueryIO.yml](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigQueryIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigQueryIO.yml?query=event%3Aschedule) |
519+
| [ StressTests Java BigTableIO ](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigTableIO.yml) | N/A | [![.github/workflows/beam_StressTests_Java_BigTableIO.yml](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigTableIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_BigTableIO.yml?query=event%3Aschedule) |
520+
| [ StressTests Java KafkaIO ](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_KafkaIO.yml) | N/A | [![.github/workflows/beam_StressTests_Java_KafkaIO.yml](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_KafkaIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_KafkaIO.yml?query=event%3Aschedule) |
521+
| [ StressTests Java PubSubIO ](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_PubSubIO.yml) | N/A | [![.github/workflows/beam_StressTests_Java_PubSubIO.yml](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_PubSubIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_PubSubIO.yml?query=event%3Aschedule) |
522+
| [ StressTests Java SpannerIO ](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_SpannerIO.yml) | N/A | [![.github/workflows/beam_StressTests_Java_SpannerIO.yml](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_SpannerIO.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_StressTests_Java_SpannerIO.yml?query=event%3Aschedule) |
505523

506524
### Other Jobs
507525

@@ -522,3 +540,8 @@ PostCommit Jobs run in a schedule against master branch and generally do not get
522540
| [ Release Nightly Snapshot Python ](https://github.com/apache/beam/actions/workflows/beam_Release_Python_NightlySnapshot.yml) | N/A | [![.github/workflows/beam_Release_Python_NightlySnapshot.yml](https://github.com/apache/beam/actions/workflows/beam_Release_Python_NightlySnapshot.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Release_Python_NightlySnapshot.yml?query=event%3Aschedule) |
523541
| [ Rotate IO-Datastores Cluster Credentials ](https://github.com/apache/beam/actions/workflows/beam_IODatastoresCredentialsRotation.yml) | N/A | [![.github/workflows/beam_IODatastoresCredentialsRotation.yml](https://github.com/apache/beam/actions/workflows/beam_IODatastoresCredentialsRotation.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_IODatastoresCredentialsRotation.yml?query=event%3Aschedule) |
524542
| [ Rotate Metrics Cluster Credentials ](https://github.com/apache/beam/actions/workflows/beam_MetricsCredentialsRotation.yml) | N/A | [![.github/workflows/beam_MetricsCredentialsRotation.yml](https://github.com/apache/beam/actions/workflows/beam_MetricsCredentialsRotation.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_MetricsCredentialsRotation.yml?query=event%3Aschedule) |
543+
| [ Beam Metrics Report ](https://github.com/apache/beam/actions/workflows/beam_Metrics_Report.yml) | N/A | [![.github/workflows/beam_Metrics_Report.yml](https://github.com/apache/beam/actions/workflows/beam_Metrics_Report.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Metrics_Report.yml?query=event%3Aschedule) |
544+
| [ GCP Security Log Analyzer ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_SecurityLogging.yml) | N/A | [![.github/workflows/beam_Infrastructure_SecurityLogging.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_SecurityLogging.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_SecurityLogging.yml?query=event%3Aschedule) |
545+
| [ Infrastructure Policy Enforcer ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml) | N/A | [![.github/workflows/beam_Infrastructure_PolicyEnforcer.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_PolicyEnforcer.yml?query=event%3Aschedule) |
546+
| [ Modify the GCP User Roles according to the infra/users.yml file ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml) | N/A | [![.github/workflows/beam_Infrastructure_UsersPermissions.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_UsersPermissions.yml?query=event%3Aschedule) |
547+
| [ Service Account Keys Management ](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml) | N/A | [![.github/workflows/beam_Infrastructure_ServiceAccountKeys.yml](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml/badge.svg?event=schedule)](https://github.com/apache/beam/actions/workflows/beam_Infrastructure_ServiceAccountKeys.yml?query=event%3Aschedule) |

‎.github/workflows/beam_PreCommit_GHA.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,7 +58,7 @@ env:
5858
jobs:
5959
beam_PreCommit_GHA:
6060
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
61-
runs-on: [self-hosted, ubuntu-20.04, main]
61+
runs-on: [self-hosted, ubuntu-20.04, small]
6262
strategy:
6363
matrix:
6464
job_name: [beam_PreCommit_GHA]

‎.github/workflows/beam_PreCommit_RAT.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ env:
5656
jobs:
5757
beam_PreCommit_RAT:
5858
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
59-
runs-on: [self-hosted, ubuntu-20.04, main]
59+
runs-on: [self-hosted, ubuntu-20.04, small]
6060
strategy:
6161
matrix:
6262
job_name: [beam_PreCommit_RAT]

‎.github/workflows/beam_PreCommit_Whitespace.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ env:
5757
jobs:
5858
beam_PreCommit_Whitespace:
5959
name: ${{ matrix.job_name }} (${{ matrix.job_phrase }})
60-
runs-on: [self-hosted, ubuntu-20.04, main]
60+
runs-on: [self-hosted, ubuntu-20.04, small]
6161
strategy:
6262
matrix:
6363
job_name: [beam_PreCommit_Whitespace]

0 commit comments

Comments
 (0)