CodeAurora Forum Support

Last modified by Lisa Lammens on 2017/11/08 17:54

Getting Started

What is CAF and where is the code used?

Code Aurora Forum (CAF), a collaborative open source project at The Linux Foundation, hosts tested open source software code needed to provide open source and upstream enablement for innovative, performance optimized, network connectivity and related ecosystems to support system on a chip (SoC) products.  It also serves as a staging area for open source code that is submitted to various upstream projects such as the Linux kernel and Android.  CAF also mirrors key upstream projects for use by the community.  CAF uses git as "the stupid content tracker".

It is important to note that only open source software is hosted on CAF.  Proprietary software and pre-built executable binary artifacts (e.g. object code, shared libraries, bytecode, or firmware) do NOT belong on CAF.  An exception is made for firmware binaries that are provided for convenience that follow the kernel.org linux-firmware guidelines.

The Linux kernel,  Android, OEM wireless handsets, tool chains such as The LLVM Compiler Infrastructure and license identifier scan tools are some of the areas using code from CAF.

How to get started on downloading code from CAF

All open source software hosted on CAF Public Projects and CAF@github Projects is freely available for download – membership and/or registration on CAF is not required.  

Open Source projects are listed here and here.  Project Admins are the maintainers of their respective projects and provide the relevant needed info for their project in the project’s wiki or README file.   

Please use https rather than SSH to clone the public repositories.  CAF public mirrors use geo-IP based DNS servers to direct default requests to the nearest server.

General technical support in the CAF environment is available 24/7/365 by submitting email to ITPeople AT codeaurora DOT org

Quick-start for CAF Members

Step 1:  Register as a Developer

Step 2:  Submit your SSH key

Step 3:  Request CAF Project setup/permissions

Step 4:  Prepare your code

Step 5: Push your code (git clone/git push) to your CAF Project

Pre-requisites for developers/engineers to contribute code to CAF

You must be a member to create CAF projects and commit code to CAF.  Approved developers and release engineers should register and create an account only after they have reviewed open source best practice guidelines.  Note: please work with your Trusted Advisor on confirming approval before registering.

Members of CAF can request support for new open source project creation, adding additional repos to projects, mirroring of third-party repositories and public projects, creating SSH keys, updates to project permissions and get help with general technical support in the CAF environment by submitting email to ITPeople AT codeaurora DOT org

Contributing members should ensure they adhere to open source best practice guidelines, license compliance and code review  to ensure the quality, consistency and appropriate due diligence of open source software code being committed to CAF.  It is considered good practice to establish a relationship with a Trusted Advisor or Compliance Officer, possibly within your developer community or company, who would be responsible to ensure open source software compliance prior to committing code to CAF.  

Guidelines for the role of a Trusted Advisor or Compliance Officer in an Open Source Community

  • Facilitate processes to handle Open Source the right way by providing best practice guidelines and/or pre-requisite training for developers and release engineers
  • Work with software developers to get their code through an approved vetting process in a timely manner 
  • Assist developers in the creation of documentation necessary to gain acceptance of their software into the software baseline
  • Work with various stakeholders to obtain permission to release software under appropriate open source license
  • Work with development teams to meet their needs and ensure process consistency across open source projects
  • Help define, implement and track processes to ensure compliance with Open Source policies

Once you confirm approval with your Trusted Advisor, you should use your Company identity/email to register as a developer on CAF.  Upon successful registration and after receiving prior approval by your Trusted Advisor, you will receive your CAF email – use of this CAF email will identify you as a CAF Member.  Registered developers should work with their Trusted Advisor or relevant Compliant Officer on appropriate email usage.

Creating SSH keys for authentication

SSH keys are used to commit code to CAF.  SSH keys offer a highly secure way of logging into a server and are best practice for authentication, allowing more security than a simple password.  SSH keys are tied to your identity (your email used on CAF) so you will need to ensure you have created or submitted SSH keys that match your identity.  Partners of members who need to access code via SSH will also need to register on CAF so their SSH keys can be stored in gitolite and their permissions validated with their registered identity.  

Using your CAF or company appropriate email identity, you should create and/or submit your SSH keys to your Developer account.

Code Guidelines -- preparing your code for CAF

Please remember that only open source software is hosted on CAF.  Proprietary software and pre-built executable binary artifacts (e.g. object code, shared libraries, bytecode, or firmware) do NOT belong on CAF.  An exception is made for firmware binaries that are provided for convenience that follow the *kernel.org linux-firmware guidelines.  To provide assurance of the code provenance of all hosted code for a CAF project, The Linux Foundation recommends bringing in existing codebases through a review system as historical commits could contain non-compliant code not present in the latest commit. 

Contributing members are strongly encouraged to work with their Trusted Advisor to receive approval on code review guidelines to ensure the quality, consistency and appropriate due diligence of open source software code being committed to CAF.

CAF projects and code must include appropriate marking guidelines:

  • All new inbound code contributions must be made using an OSI-approved open source license specified for the Project within the “LICENSE” file
  • All new inbound code contributions must also be accompanied by a Developer Certificate of Origin sign-off
  • All outbound code will be made available under the applicable Project license
  • Documentation will be received and made available by the Project under the Creative Commons Attribution 4.0 International License
  • If you have a signed Contributor License Agreement (CLA) that provides for copyright assignment to The Linux Foundation, please use the following format:
    • Copyright (c) {year(s)}, The Linux Foundation. All rights reserved.
  • Please respect copyright attributions -- DO NOT REMOVE OR MODIFY THIRD PARTY COPYRIGHT OR LICENSE MARKINGS
  • Please work with your Trusted Advisor if you have questions regarding appropriate usage

*Firmware Binaries "provided for convenience":

This is the old way firmware was included in the kernel: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/firmware (legacy support)

These days, it goes in the linux-firmware repo here: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/

The README has some language about the restrictions: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/README

And this file lists all the licenses for the firmware blobs: https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/WHENCE

Request to create a new CAF project/area

Often, CAF project/area hierarchies reflect the structure of repositories on an internal git server that a developer may already use, although that structure is not mandatory.  Developers should select an overall hierarchy prior to creating new repositories or mirroring third-party repositories on CAF, so that the hierarchy reflects the long-term desired structure for the project.

To request a new CAF project/area where you can push your open source contributions, please see Project Requirements.  Once the project info is received, ITPeople will open a ticket to set up your project, your wiki template and configure your permissions.  Please be sure to use your appropriate email identity when communicating with ITPeople.

Note that when a new repository is created on CAF it will not be visible until you have pushed content to that repository.

Request access to a CAF project/area

Please send email to ITPeople AT codeaurora DOT org with the following information:

  1. Your Name
  2. Your Email Address
  3. Your CAF ID (login ID)
  4. Your SSH-Key fingerprint (NOT your SSH key)
  5. The Repo/Project Path URL
  6. Be sure to Cc: the Project Admins for approvals

Mirroring

The first step towards making open source contributions/modifications to an upstream or CAF project is to typically  mirror the upstream project to the appropriate project on CAF.  Your mirroring request should first be vetted by your Trusted Advisor.  Once approved, the Project Admin for the respective project should send the email request for the mirroring setup to ITPeople AT codeaurora DOT org.  ITPeople will then set up the mirror. Please use the following mirror request format for consistency:

  1. CAF Project: The project name on CAF for this request
  2. URL mirrored from: The URL for the external site to be mirrored
  3. URL to CAF git project: The path where the code will be mirrored to
  4. Re-synced: Frequency of mirror re-sync (daily, weekly, one-time)
  5. POC: Contact name and email (be sure to use your appropriate identity)
  6. Approval Reference: URL of approved Trusted Advisor ticket or email copy of approval

Once the mirror is setup, the developer should be able to mirror the CAF repo to an internal server repo and begin making modifications on top of the upstream baseline.

Pushing code to a CAF Project

Push to CAF is defined here as the act of pushing open source code to a pre-determined repo on an existing project at CAF.  To provide assurance of the code provenance of all hosted code for a CAF project, the Linux Foundation recommends bringing in existing codebases through a code review system as historical commits could contain non-compliant code not present in the latest commit.

Note that when a new repository is created on CAF it will not be visible until you have pushed content to that repository.

Every project on CAF has a Project Admin who authorizes the write permissions with ITPeople for their respective project and repos.  If you are requesting push access to an existing project or repo, you will need authorization and approval from the respective Project Admin.  These project and repo access permissions will be stored and managed in a git config file for access by ITPeople.

There are pre-requisites and several steps in this process as follows.

  1. Relevant roles for Trusted Advisor and Release Engineer for the project have been established and approved (see above reference  Guidelines for the role of a Trusted Advisor in an Open Source Community)
  2. Completion of relevant training for CAF distribution process
  3. Write permission approval and set up has been completed by ITPeople
  4. OSS Code has been vetted and approved by the project's Trusted Advisor
  5. Project has been approved and set up has been completed by ITPeople

 

Note that all code to be upload to CAF needs prior approval by your appointed “Trusted Advisor” or engineering team member who will be responsible to ensure your code has been appropriately vetted and approved using the following guidelines:

  • Appropriate OSS license is confirmed and in compliance
  • DCO (signed-off-by) or Contribution Agreement has been completed.
  • Code is free of executable binary artifacts (e.g. object code, executables, shared libraries, bytecode, or firmware. An exception is made for firmware binaries that are provided for convenience that follow the kernel.org linux-firmware guidelines.)
  • Code is confirmed open source
  • Your Trusted Advisor has approved the release

Once approval has been confirmed, code is ready to be pushed to CAF.  Below can be used as reference:

# 1. Clone the repo from your internal Git server

git clone <internal_git_server>/<internal_repository_name> -b <internal_branch_name>

# 2. Push a commit (and its ancestors) to CAF

# Note the caret (^) after the approved_git_tag. This is Git's way of dereferencing a tag to the commit ID that it points to.

git push ssh://gitolite3@git.codeaurora.org:22/<CAF_area>/<CAF_repository_name> <approved_git_tag>^:refs/heads/<CAF_branch_name>

# 3. Push a Git tag to CAF

git push ssh://gitolite3@git.codeaurora.org:22/<CAF_area>/<CAF_repository_name> <approved_git_tag>

#4. Push a branch to CAF repo

In Git 1.7.0 and later, you can checkout a new branch:

git checkout -b <branch>

Edit files, add and commit. Then push with the -u option:

git push -u origin <branch>

Note that "CAF_area" is the project/area where the repository resides on CAF. Also, the internal and CAF repository names may be the same as well as the internal and CAF branch names. However, this is just a matter of convenience and not a requirement.

You may want to consider using automation and service accounts if you need to regularly commit code to CAF.

#5. Push a branch from an existing repository to a new CAF repo

If you are moving a repository from an existing location, you will need to add the CAF remote to your local clone of that repository. 

To do that, add the remote using the git command line (NOTE that the git@ tells it to use ssh, so the command should not need ssh://):

git remote add <remote_name> git@git.codeaurora.org:external/<CAF_area>/<CAF_repository_name>

Then you will be able to push existing branches/tags to this remote, for example:

git push -u <remote_name> <branch_name>

If CAF is the new home for this source code, you should probably do a fresh clone from codeaurora.org and start working in that new location so everything is setup properly.  

Alternatively this can be done without adding a local remote, by pushing directly to the URL:

git push git@git.codeaurora.org:external/<CAF_area>/<CAF_repository_name> <branch_name>

If you want to push to a differently named branch than the original repository:

git push git@git.codeaurora.org:external/<CAF_area>/<CAF_repository_name> <old_branch_name>:<caf_branch_name>

Making a Private Project Public

The Project Admin should email ITPeople AT codeaurora DOT org and ask for the project to be made public.  There are many options of how this could be done, but these are the three most commonly used:

  • projects/repos are made public in the same location they existed as private
  • projects/repos are moved to a new location to be made public
  • project owners can push the private commits to a public repo to make them public

After the second two options, the private locations should then be removed after all commits are confirmed to exist in the public location.

CAF@github Projects (Coming Soon)

*GitHub-specific instructions for onboarding codebases*

1. After ITPeople creates a given repository and sets up permissions under the CAF Org, an initial committer can fork the repository to their own account.  

2. Clone the repository locally, which will contain an initial  commit. We recommend at this point you create a topic branch locally, rather than working directly off the master branch.

3. You will need to copy or rebase your incoming code into the repository and commit it.  Either way, the commit(s) must pass code provenance requirements:

  • must have a DCO signature in  the commit message (git commit -s),
  • must use compliant licenses, and
  • may not contain any executable binary artifacts (e.g. object code, executables, shared libraries, bytecode, or firmware). An exception is made for firmware binaries that are provided for convenience that follow the *kernel.org linux-firmware guidelines.  

         * Copying the working directory is the most straightforward, and does not bring in any history. Make sure to copy all files including hidden files (files starting with “.”).  Add and commit the files with a DCO sign-off.

         * History can be brought in via rebasing or cherry-pick the entire history, provided the above conditions are met, but this process is more involved and left as an exercise for the committer.  Note, regardless of how this is done, the commits hashes will not match earlier versions of the repository.

4. Push the code to the corresponding topic branch on your GitHub fork.

5. Create a pull request from your topic branch to the master branch on project’s repository, which will validate the commit(s) with a DCO check and any CI verify jobs in place.

6. Someone other than that contributor will have to review the pull request and merge it, as we enforce independent reviews on all contributions.

Tips

a good command for testing access:

ssh git@git.codeaurora.org info <CAF_area>/<CAF_project>/<CAF_repo_name>

corresponding output:

hello <CAF_username>, this is git@pdx-caf-gitolite running gitolite3 3.6.6-1.el6 on git 2.13.2

R W <CAF_area>/<CAF_project>/<CAF_repo_name>

You can push between branches of different names, by substituting <branch_name> as <local_branch>:<remote_branch>.


Helpful GitHub References

Subversion to Git Migration

Support for Subversion Clients

GitHub cheat sheet (multi-language)

Help & Support Links

Tags:
Created by stewrat on 2010/09/01 07:35
   
This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 7.4.3 - Documentation