SCC

Brasil

os cloud gurus

Software Cloud Consulting

Your software development, cloud, consulting & shoring company

Working with Git Submodules in Code Pipeline


By Wolfgang Unger


If you use Git Submodules in your Project you won't be able to use this in CodePipeline out of the box,
meaning using the Source Step as for a simple Git Project.
There are some additional steps required, this tutorial will show you how to proceed.
Your Main Github Project will remain the Source of your Pipeline ( probably defined as Source Step in your Pipeline)
For the Git Sub-Module you will have to add a CodeBuild Project and a Buildspec Yaml.
If you setup your project manually on the UI, you would have to create a CodeBuild project and insert the build commands ( to a new Builspec Yaml)
This Buildspec Yaml would be look like This
version: 0.2
phases:
  install:
    runtime-versions:
python: 3.9
    commands:
      - git remote add github https://path_to_main_repo.git
      - git fetch github
      - git branch
      - git checkout -f github/main
      - git submodule init
      - git submodule update --recursive
  build:
    commands:
      - python3.9 optional_commands.py

Since we are using CDK to deploy our pipeline ( of course !) we will make the changes in our cdk code
Our Synth Step - and actually each step which will need the git submodule files, meaning for example an integration test step needs
this file will also need this change - will get an additional optional parameter:
partial_build_spec

which can be added for example after the input parameter

synth_step = pipelines.CodeBuildStep(
  "Synth",
  input=git_input,
  partial_build_spec=self.get_source_buildspec(),
 .....

Since we might need to add this CodeBuild Project for more Steps in our project we will not add the code inline here,
but rather create a own method to be able to call this code each time a step needs the submodule

def get_source_buildspec(self):
  build_spec = aws_codebuild.BuildSpec.from_object(
   {
    "version": "0.2",
    "phases": {
     "install": {
      "runtime-versions": {"python": "3.9"},
      "commands": [
       "git init",
       "git remote add github https://path_to_your_main_repo.git",
       "git fetch github",
       "git branch",
       "git checkout -f github/main",
       "git status",
       "git submodule init",
       "git submodule update",
    ],
     },
    "build": {"commands": ["your optional commands"]},
    },
   }
)
return build_spec

Now your CodeBuild Steps will checkout the git submodule and you can use the files for whatever you want to do
cdk synth, call tests, etc
Have fun !

  • Back to Blog Overview
  • Autor


    ...

    Wolfgang Unger

    AWS Architect & Developer

    6 x AWS Certified

    1 x Azure Certified

    A Cloud Guru Instructor

    Certified Oracle JEE Architect

    Certified Scrum Master

    Certified Java Programmer

    Passionate surfer & guitar player