CodeAurora Forum Support

Last modified by Lisa Lammens on 2018/05/30 14:40

Getting Started

What is CAF and where is the code used?

Code Aurora Forum (CAF), a collaborative open source project at The Linux Foundation, hosts tested open source software code needed to provide open source and upstream enablement for innovative, performance optimized, network connectivity and related ecosystems that support system on a chip (SoC) products.  It also serves as a staging area for open source code that is submitted to various upstream projects such as the Linux kernel and Android.  CAF also mirrors key upstream projects for use by the community.  CAF uses git as "the stupid content tracker".

It is important to note that only open source software is hosted on CAF.  Proprietary software and pre-built executable binary artifacts (e.g. object code, shared libraries, bytecode, or firmware) do NOT belong on CAF.  An exception is made for firmware binaries that are provided for convenience that follow the kernel.org linux-firmware guidelines.

The Linux kernel,  Android, OEM wireless handsets, Automotive, mobile computing, IoT and Security, tool chains such as The LLVM Compiler Infrastructure and license identifier scan tools are some of the areas using code from CAF.

How to get started on downloading code from CAF

All open source software hosted on CAF Public Projects and repos at CAF@GitHub is freely available for download – membership and/or registration on CAF is not required.  

Open Source projects are listed here and GitHub repos are here.  Project Admins are the maintainers of their respective projects and provide the relevant needed info for their project or repos in the project’s wiki or README file.   

Please use https rather than SSH to clone the public repositories.  CAF public mirrors use geo-IP based DNS servers to direct default requests to the nearest server.  Developers can configure their systems manually to direct repo requests to one of the three servers that works best for them.

General technical support in the CAF environment is available 24/7/365 by submitting email to ITPeople AT codeaurora DOT org

Quick-start for CAF Developers/Members

Step 1:  Register as a Developer

Step 2:  Submit your SSH key

Step 3:  Request CAF Project setup/permissions

Step 4:  Prepare your code

Step 5: Push your code (git clone/git push) to your CAF Project

Quick-start for downloading code from private repo

Step 1:  Register as a Developer

Step 2:  Submit your SSH key

Step 3:  Request private CAF project/repo access

CAF@GitHub

CAF maintains many open source repos on CAF@GitHub.  GitHub-specific instructions for onboarding codebases are available at the CAF@GitHub Support Page. The general CAF requirements listed here also apply to the repos on CAF@GitHub.

IRC

Please see the ZNC for details on the IRC bouncer run by CAF.

Pre-requisites for developers/engineers to contribute code to CAF

Contributing members should ensure they adhere to open source best practice guidelines, license compliance, export control  and code review  to ensure the quality, consistency and appropriate due diligence of open source software code being committed to CAF.  It is considered good practice to establish a relationship with a Trusted Advisor or Compliance Officer, possibly within your developer community or company, who would be responsible to ensure open source software compliance prior to committing code to CAF.  

You must be a member to create CAF projects and commit code to CAF.  Approved developers and release engineers should register and create an account only after they have reviewed open source best practice guidelines.  Note: please work with your company's Trusted Advisor or Compliance Officer to confirm approval before registering.

Members of CAF can request support for new open source project creation, adding additional repos to projects, mirroring of third-party repositories and public projects, creating SSH keys, updates to project permissions and get help with general technical support in the CAF environment by submitting email to ITPeople AT codeaurora DOT org.  

Guidelines for the role of a Trusted Advisor or Compliance Officer in an Open Source Community

  • Facilitate processes to handle Open Source the right way by providing best practice guidelines and/or pre-requisite training for developers and release engineers
  • Work with software developers to get their code through an approved vetting process in a timely manner 
  • Assist developers in the creation of documentation necessary to gain acceptance of their software into the software baseline
  • Work with various stakeholders to obtain permission to release software under appropriate open source license
  • Work with development teams to meet their needs and ensure process consistency across open source projects
  • Help define, implement and track processes to ensure compliance with Open Source policies

Once you confirm approval with your Trusted Advisor or Compliance Officer, you should use your Company identity/email to register as a developer on CAF.  Upon successful registration and after receiving prior approval by your Trusted Advisor, you will receive your CAF email – use of this CAF email will identify you as a CAF Member.  Registered developers should work with their Trusted Advisor or relevant Compliant Officer on appropriate email usage.

Creating your SSH key for authentication

SSH keys are used to commit code to CAF.  SSH keys offer a highly secure way of logging into a server and are best practice for authentication, allowing more security than a simple password.  SSH keys are tied to your identity (your email used on CAF) so you will need to ensure you have created or submitted an SSH key that matches your identity.  Partners of members who need to access code via SSH will also need to register on CAF so their SSH key can be stored in gitolite and their permissions validated with their registered identity.  Partner identities should match company email.

Using your CAF or company appropriate email identity, you should create and/or submit your SSH key to your Developer account.  Unregistered keys are not allowed and will be deleted.

Code Guidelines -- preparing your code for CAF

Please remember that only open source software is hosted on CAF.  Proprietary software and pre-built executable binary artifacts (e.g. object code, shared libraries, bytecode, or firmware) do NOT belong on CAF.  An exception is made for firmware binaries that are provided for convenience that follow the *kernel.org linux-firmware guidelines.  To provide assurance of the code provenance of all hosted code for a CAF project, The Linux Foundation recommends bringing in existing codebases through a review system as historical commits could contain non-compliant code not present in the latest commit. 

Contributing members are strongly encouraged to work with their Trusted Advisor/Compliance Officer to receive approval on code review guidelines to ensure the quality, consistency and appropriate due diligence of open source software code being committed to CAF.

CAF projects and code must include appropriate marking guidelines:

  • All new inbound code contributions must be made using an OSI-approved open source license specified for the Project within the License file
  • All new inbound code contributions must also be accompanied by a Developer Certificate of Origin sign-off; more info on DCO 
  • All outbound code will be made available under the applicable Project license
  • Documentation will be received and made available by the Project under the Creative Commons Attribution 4.0 International License
  • If you have a signed Contributor License Agreement (CLA) that provides for copyright assignment to The Linux Foundation, please use the following format:
    • Copyright (c) {year(s)}, The Linux Foundation. All rights reserved.
  • Please respect copyright attributions -- DO NOT REMOVE OR MODIFY THIRD PARTY COPYRIGHT OR LICENSE MARKINGS
  • Please work with your Trusted Advisor if you have questions regarding appropriate usage
Additional Requirements

If you are creating a new CAF repository from scratch, we require that your repository includes:

We also strongly suggest you use some of the suggest tools to enforce these requirements.

*Firmware Binaries "provided for convenience":

This is the old way firmware was included in the kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/firmware (legacy support)

These days, it goes in the linux-firmware repo here: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/

The README has some language about the restrictions: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/README

And this file lists all the licenses for the firmware blobs: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/WHENCE

Request to create a new CAF project/area

Often, CAF project/area hierarchies reflect the structure of repositories on an internal git server that a developer may already use, although that structure is not mandatory.  Developers should select an overall hierarchy prior to creating new repositories or mirroring third-party repositories on CAF, so that the hierarchy reflects the long-term desired structure for the project.  If your project is expected to have many repos, you may want to consider using folders to better categorize them... please see git repos for qoriq and imx projects for reference.

To request a new CAF project/area where you can push your open source contributions, please see Project Requirements.  Once the project info is received, ITPeople will open a ticket to set up your project, your wiki template and configure your permissions.  Please be sure to use your appropriate email identity when communicating with ITPeople.

Note that when a new repository is created on CAF it will not be visible until you have pushed content to that repository.

Request access or permissions to a CAF project/area

You will need to Register as a Developer and  Submit your SSH key for access.  Please send email to ITPeople AT codeaurora DOT org with the following information:

  1. Your Name
  2. Your Email Address
  3. Your CAF ID (login ID)
  4. Your SSH-Key fingerprint (NOT your SSH Key which should always remain private)
  5. The Repo/Project Path URL
  6. Be sure to Cc: the Project Admins for all permissions approvals
  7. Please specify your request for Write Access or Permission to Access a Private Project

Mirroring

The first step towards making open source contributions/modifications to an upstream or CAF project is to typically  mirror the upstream project to the appropriate project on CAF.  Your mirroring request should first be vetted by your Trusted Advisor.  Once approved, the Project Admin for the respective project should send the email request for the mirroring setup to ITPeople AT codeaurora DOT org.  ITPeople will then set up the mirror. Please use the following mirror request format for consistency:

  1. CAF Project: The project name on CAF for this request
  2. URL mirrored from: The URL for the external site to be mirrored
  3. URL to CAF git project: The path where the code will be mirrored to
  4. Re-synced: Frequency of mirror re-sync (daily, weekly, one-time)
  5. POC: Contact name and email (be sure to use your appropriate identity)
  6. Approval Reference: URL of approved Trusted Advisor ticket or email copy of approval

Once the mirror is setup, the developer should be able to mirror the CAF repo to an internal server repo and begin making modifications on top of the upstream baseline.

Pushing code to a CAF Project

Push to CAF is defined here as the act of pushing open source code to a pre-determined repo on an existing project at CAF.  To provide assurance of the code provenance of all hosted code for a CAF project, The Linux Foundation recommends bringing in existing codebases through a code review system as historical commits could contain non-compliant code not present in the latest commit.

Note that when a new repository is created on CAF it will not be visible until you have pushed content to that repository.

Every project on CAF has a Project Admin who authorizes the write permissions with ITPeople for their respective project and repos.  If you are requesting push access to an existing project or repo, you will need authorization and approval from the respective CAF Project Admin.  These project and repo access permissions will be stored and managed in a git config file by ITPeople.

There are pre-requisites and several steps in this process as follows.

  1. Relevant roles for Trusted Advisor and Release Engineer for the project have been established and approved (see above reference  Guidelines for the role of a Trusted Advisor in an Open Source Community)
  2. Write permission approval and set up has been completed by ITPeople
  3. OSS Code has been vetted and approved by the project's Trusted Advisor
  4. Project has been approved and set up has been completed by ITPeople

Note that all code to be upload to CAF needs prior approval by your appointed “Trusted Advisor” or engineering team member who will be responsible to ensure your code has been appropriately vetted and approved using the following guidelines:

  • Appropriate OSS license is confirmed and in compliance
  • DCO (signed-off-by) or Contribution Agreement has been completed.
  • Code is free of executable binary artifacts (e.g. object code, executables, shared libraries, bytecode, or firmware. An exception is made for firmware binaries that are provided for convenience that follow the kernel.org linux-firmware guidelines.)
  • Code is confirmed open source
  • Your Trusted Advisor has approved the release

Once approval has been confirmed, code is ready to be pushed to CAF.  Below can be used as reference:

# 1. Clone the repo from your internal Git server

git clone <internal_git_server>/<internal_repository_name> -b <internal_branch_name>

# 2. Push a commit (and its ancestors) to CAF

# Note the caret (^) after the approved_git_tag. This is Git's way of dereferencing a tag to the commit ID that it points to. Your CAF Area is whole repo list (e.g., external/shortname that you requested in your project setup). The CAF Repository name will be the full name listed under your project; if you have a path in your repo name you will need to include the folder path in the name.

git push ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name> <approved_git_tag>^:refs/heads/<CAF_branch_name>

# 3. Push a Git tag to CAF

git push ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name> <approved_git_tag>

#4. Push a branch to CAF repo

In Git 1.7.0 and later, you can checkout a new branch:

git checkout -b <branch>

Edit files, add and commit. Then push with the -u option:

git push -u origin <branch>

Note that "CAF_area" is the project/area where the repository resides on CAF. Also, the internal and CAF repository names may be the same as well as the internal and CAF branch names. However, this is just a matter of convenience and not a requirement.

You may want to consider using automation and service accounts if you need to regularly commit code to CAF.

#5. Push a branch from an existing repository to a new CAF repo

If you are moving a repository from an existing location, you will need to add the CAF remote to your local clone of that repository. 

To do that, add the remote using the git command line (NOTE that the git@ tells it to use ssh, so the command should not need ssh://):

git remote add <remote_name> ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name>

Then you will be able to push existing branches/tags to this remote, for example:

git push -u <remote_name> <branch_name>

If CAF is the new home for this source code, you should probably do a fresh clone from codeaurora.org and start working in that new location so everything is setup properly.  

Alternatively this can be done without adding a local remote, by pushing directly to the URL:

git push ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name> <branch_name>

If you want to push to a differently named branch than the original repository:

git push ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name> <old_branch_name>:<caf_branch_name>

#6. Pushing a mirror of a repository

Typically, mirroring is performed by the CAF Admins upon request, however in some cases project administrators may need to mirror a repository to CAF. If this is the case, the commands below serve as an example:

git clone --mirror git@example.com/upstream-repository.git
cd upstream-repository.git
git push --mirror ssh://gitolite3@git.codeaurora.org:22/external/<CAF_shortname>/<CAF_repository_name>

#7. Pushing a patch to the patch area of a repository

Make sure you have requested a patch area for your project (example: source.codeaurora.org/external/sba/<CAF_shortname>_patches). If that is configured you should be able to push to the master branch of your patch repository to make the new patches available in the web view.

git push ssh://gitolite3@git.codeaurora.org:22/external/sba/<CAF_shortname>_patches.git master

Making a Private Project Public

The Project Admin should email ITPeople AT codeaurora DOT org and ask for the project to be made public.  There are many options of how this could be done, but this is the most common way used:

  • a new public project with public repos is created in a new location
  • project owners can push the private commits to a public repo to make them public

After this is complete, the private locations should then be removed after all commits are confirmed to exist in the public location.

Tips & Troubleshooting

a good command for testing access:

ssh git@git.codeaurora.org info <CAF_area>/<CAF_shortname>/<CAF_repo_name>

For example:

ssh git@git.codeaurora.org info external/qostg/lid

corresponding output:

hello <CAF_username>, this is git@pdx-caf-gitolite running gitolite3 3.6.6-1.el6 on git 2.13.2

R W <CAF_area>/<CAF_project>/<CAF_repo_name>

You can push between branches of different names, by substituting <branch_name> as <local_branch>:<remote_branch>.

A command for getting more information on failures:

ssh -T -v gitolite3@git.codeaurora.org

Push issues:

  • verify you have permissions/access to the repo using the above testing access command
  • verify that your SSH key is correct (and that it was not changed/updated or that you have more than 1 in your account; contact ITPeople AT codeaurora DOT org for help) 
  • verify that you have registered on CAF
  • verify that your IT does not block port 22 
  • verify your push command is correct and using ssh (make sure you are connecting as the 'git' user)
  • verify APAC mirror is updating by cloning the US mirror

Clone issues:

  • verify the repo is not empty (empty repos cannot be cloned until there is an initial push)
  • verify clone path is correct
  • verify APAC mirror is updating by cloning the US mirror

Private Projects:

  • verify that you have registered on CAF
  • verify you have permissions/access to the repo using the above testing access command
  • verify that your SSH key is correct (and that it was not changed/updated or that you have more than 1 in your account; contact ITPeople AT codeaurora DOT org for help)

Mirror issues:

  • if you aren't seeing the mirror updates, please wait 15 minutes for updates and double check before sending mail to ITPeople AT codeaurora DOT org for help

Helpful Git References

Subversion to Git Migration

Support for Subversion Clients

Git cheat sheet (multi-language)

GitHub Resources

Tags:
Created by stewrat on 2010/09/01 07:35
   
This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 7.4.3 - Documentation