Admin |
Main /
EECSRepoThe purpose of the EECS Repo project is to create a facility that allows UCB and non-UCB participants to access shared version control resources for files such as papers and software development. The motivation is that there is a need for students, faculty and staff to collaborate on shared files in a secure manner and avoid having non-backed up state on laptops etc. By providing a common Source Code Management (SCM) infrastructure, we can avoid costly duplication of effort. As of October, 2012, EECS Repo supports Subversion (aka SVN) and Git. Quick StartTo get started:
$ cd /tmp $ svn co https://repo.eecs.berkeley.edu/svn/users/YOURNAME/test1 Checked out revision 0. or for a git repository: $ cd /tmp $ git clone https://repo.eecs.berkeley.edu/svn/users/YOURNAME/test1.git Cloning into 'test1'... The Location of the SVN RepositoryThe Subversion command to check out a repository is: In EECS Repo, an SVN repository is either a personal space or a non-personal space. A SVN repository can be accessed either with named accounts or (if enabled) anonymously. There are personal SVN repositories that have the following repositoryURL: There are non-personal SVN repositories: Hint: To see the path of a repository, go Repositories -> Access and click on the repository name. Subversion Clients
The Location of the Git RepositoryThe git command to check out a repository is: In EECS Repo, a Git repository is either a personal space or a non-personal space. A Git repository can be accessed either with named accounts or (if enabled) anonymously. There are personal Git repositories that have the following repositoryURL: There are non-personal Git repositories: Hint: To see the path of a repository, go Repositories -> Access and click on the repository name. Git Clients
Repository SuggestionsWhen deciding on a layout, things to consider include
PapersIf a group is regularly collaborating on papers, then create a repository for that group and then add subdirectories by year and then by papers. For example, the fictitious Professor Turing might have a paper called SoftwareTypically, in Subversion, a ProposalsEach proposal usually has a slightly different set of collaborators. Each proposal may want to be a separate repository in the user space of a faculty or staff member. Individual users can be granted read-only or read-write access to the repository. Small DatasetsA handful of small datasets (a few KB to a few GB) can easily be shared by creating a repository to contain them. If the data is public, you can create the repository with anonymous read-only access. If you have more than just a couple of datasets or if it is more data than would fit onto a CD or DVD, this is probably not the best way to distribute it. Also, be sure to review the UCB policies before hosting public data. EECS Repo OverviewEECS Repo uses http://www.insanefactory.com/if-svnadmin/ iF.SVNAdmin. The iF.SVNAdmin package uses Subversion's path-based authorization. There are a number of terms:
Local users can create accounts for external users to access this system. External users are named by their e-mail address, for example user@mydomain.com . Local users can control read/write access by other local and external users for repositories in spaces that they administer.
The key take away points: There are personal and non-personal spaces that can contain multiple repositories; every local user is given a personal space that they administer. Space administrators can create and manage repositories including granting access to repositories by local and non-local users. Local users can register the e-mail address of external users with the repository system in order to enable collaboration. "We intended repo project spaces as a way to manage multiple related repos whether for a specific project or for a research group or center which may have many multiple projects each with one or more repos each. So, if <Requested_Space> existed as a project within such a center or research group, then it would make more sense to create a repo space for that center or group and place <Requested_Space> within that center/group's space, especially if <Requested_Space> has only a single git repo. But if you anticipate lots of contributors from outside that group or center or if you anticipate that <Requested_Space> will grow beyond a single repo to having multiple sub-projects in their own separate repos, then it makes sense to create a separate space for <Requested_Space> now, even if it only has a single repo now." --Jon Kuroda, EECS Research EECS Repo TasksGetting started: how to create a repositoryBelow we create a new user repository called 1. Log in to https://repo.eecs.berkeley.edu with your EECS LDAP credentials. From that page you can
or or
Click on "Log in and administer repositories" and use your EECS LDAP username and password. 2. From that page you can
Hint: Non-personal space names are shared amongst all users. Typically, spaces are created for large, long term research projects that will have multiple repositories. If a project is a short term project, say collaboration on a class project, then consider creating repositories in a user space and granting access to collaborators. Or if the project is a sub-element of some larger research project such as 'ParLab' or 'AMPLab' which already has a named repository space, consider having a repository created within that space. We are going to create a repository called 3. Create a repository in the user space called 4. Create a new group called 5. Add yourself to the group:
7. Add the external user to the group:
8. Set up the permissions for the group.
In the screenshot below, we typed a 9. To view the URL of the ptpapers repository, go to Repositories -> Access and click on the access path 10. If your login on your local machine is the same as the account on the EECS Repo server, then you can access the repository with: bash-3.2$ svn co https://repo.eecs.berkeley.edu/svn/users/cxh/ptpapers/ Error validating server certificate for 'https://repo.eecs.berkeley.edu:443': - The certificate is not issued by a trusted authority. Use the fingerprint to validate the certificate manually! Certificate information: - Hostname: repo.eecs.berkeley.edu - Valid: from Wed, 22 Aug 2012 00:00:00 GMT until Sat, 22 Aug 2015 23:59:59 GMT - Issuer: InCommon, Internet2, US - Fingerprint: d3:65:79:fe:3c:ee:79:e4:6b:5d:2a:6c:dd:06:f4:05:80:ca:cb:ec (R)eject, accept (t)emporarily or accept (p)ermanently? p Authentication realm: <https://repo.eecs.berkeley.edu:443> UCB/EECS Subversion Repository Password for 'cxh': Checked out revision 0. The "Error validating server certificate" message is because the CA is not in the OpenSSH list of CAs. See svn: Certificate is not issued by a trusted authority To test access for the external user, use: bash-3.2$ svn co --username cxbrooks@gmail.com https://repo.eecs.berkeley.edu/svn/users/cxh/ptpapers/ Authentication realm: <https://repo.eecs.berkeley.edu:443> UCB/EECS Subversion Repository Password for 'cxbrooks@gmail.com': Checked out revision 0. bash-3.2$ One tricky issue is that Subversion tends to cache credentials.
If the external user password does not work, then try moving Git1. Create a Git repository:
2. Allow the
In the screenshot below, we typed a 3. To view the URL, go to Repositories -> Access and click on the access path 4. To clone the Git repository, use: bash-3.2$ git clone https://repo.eecs.berkeley.edu/git/users/cxh/gitTest.git Cloning into 'gitTest'... Username: cxh Password: warning: You appear to have cloned an empty repository. bash-3.2$ SSH keys and GitGit can use SSH Keys, to upload your key, use Services -> Upload SSH Key Below is a sample run using SSH and Git. bash-3.2$ git clone repoman@repo.eecs.berkeley.edu:users/cxh/gitTest.git Cloning into 'gitTest'... warning: You appear to have cloned an empty repository. bash-3.2$ cd gitTest bash-3.2$ echo "This is a test" > README.txt bash-3.2$ git add README.txt bash-3.2$ git commit -m "A test" README.txt [master (root-commit) 978d6ef] A test 1 files changed, 1 insertions(+), 0 deletions(-) create mode 100644 README.txt bash-3.2$ git push Note: Very first git push per repository MUST be - git push -u origin master Note that the svn server on repo.eecs.berkeley.edu does not support ssh. The svn server on repo.eecs is using the Email NotificationEmail Notification upon commit can be turn ON or OFF per repository, by going to Access/Notify page. Select all repositories from the list that required email notification and hit the Enable button. Alternatively, you can select one or more repositories and disable the notification. Once the repository is set with Email Notification ON, select the repository and go to repository Access-Path page. Use bottom section to assign/unassign users to the email list. Only users in the list will receive the email notifications. External User: Password ResetThis is only for external users who are not in the UCB EECS LDAP database.
External User: Forgot User NameExternal account usernames are email addresses of the format External account users who do not remember their account name should contact the person who created their account. Or, see How do I contact someone about this facility? Anonymous Read-Only AccessThe To make a repository anonymously accessible, add the
Hints: Note that Anonymous repositories cannot be accessed read/write. Also, making a papers repository anonymously readable is ill-advised. To access the repository anonymously: bash-3.2$ svn co https://repo.eecs.berkeley.edu/svn-anon/projects/embedded/papers Checked out revision 0. bash-3.2$ RolesUsers may have the different roles assigned to them:
To assign a role, got to Users -> Space Permissions. A user may have 0 or more roles in a repository. The roles cover the entire space. Initializing using svnsyncThe Note that the new repository must have no revisions. In the example below, we create a new user space repository, add a user to the list of SVN Syn users and then initialize the repository 1. Create a new repository:
2. Add a user to the list of SVN Sync Users (This step is only necessary if a user other than the user who created the repository will be running svnsync):
Note that this has to be done for each repository that is to be synced. 3. As a test, check out the initial revision. Note: do not check in any changes, svnsync requires an empty destination repository bash-3.2$ svn co https://repo.eecs.berkeley.edu/svn/projects/chess/ActorBook/ You may be prompted to save your password: Authentication realm: <https://repo.eecs.berkeley.edu:443> UCB/EECS Subversion Repository Password for 'cxh': ----------------------------------------------------------------------- ATTENTION! Your password for authentication realm: <https://repo.eecs.berkeley.edu:443> UCB/EECS Subversion Repository can only be stored to disk unencrypted! You are advised to configure your system so that Subversion can store passwords encrypted, if possible. See the documentation for details. You can avoid future appearances of this warning by setting the value of the 'store-plaintext-passwords' option to either 'yes' or 'no' in '/home/cxh/.subversion/servers'. ----------------------------------------------------------------------- Store password unencrypted (yes/no)? The above message occurred when running svn on the command line on a Linux machine. See Caching Credentials for information on using KWallett under Linux. If KWallett is not set up, the consider saying "no" to storing the password unencrypted. Mac OS X and Windows users should not see this message, the password is not stored in clear text. 4. Initialize the repository.
The syntax is bash-3.2$ svnsync initialize https://repo.eecs.berkeley.edu/svn/projects/chess/ActorBook https://source.eecs.berkeley.edu/svn/chess/ActorBook bash-3.2$ This command sets up properties in the destination repository. Hint: If you get a message like 5. Sync the repository bash-3.2$ svnsync sync https://repo.eecs.berkeley.edu/svn/projects/chess/ActorBook Transmitting file data ............................................... Committed revision 1. Copied properties for revision 1. Transmitting file data .. Committed revision 2. Copied properties for revision 2. bash-3.2$ 6. Check out the destination: bash-3.2$ cd /tmp bash-3.2$ svn co https://repo.eecs.berkeley.edu/svn/projects/chess/ActorBook A ojdcheck/LICENSE A ojdcheck/build-jazzy.xml A ojdcheck/README.rst ... A ojdcheck/com.github.ojdcheck.jazzy/jar/LICENSE.txt A ojdcheck/com.github.ojdcheck.jazzy/jar/jazzy-core.jar A ojdcheck/com.github.ojdcheck.jazzy/jar/CONTRIBUTORS.txt A ojdcheck/com.github.ojdcheck.jazzy/jar/README.txt Checked out revision 2. bash-3.2$ 7. You may want to check out the old repository and do a diff. 8. Then move the old repository to a different location so that users do not update the old repository. Initializing using git remoteIf you have a git repository, residing on the remote server and would like to bring it in, the following procedure can be used: 1. login to repo.eecs.berkeley.edu 2. create new repository, set permissions, etc 3. Clone the original repository from the remote server 4. on command line, change directory to that repository 5. execute: $git remote add <friendly_name_of_remote_repository> repoman@repo.eecs.berkeley.edu:users{projects}/<user_name{project_name}>/<new_repository> Or (via https) $git remote add <friendly_name_of_remote_repository> https://repo.eecs.berkeley.edu/git/users{projects}/<user_name{project_name}>/<new_repository> And then $git push <friendly_name_of_remote_repository> master NOTE: <friendly_name_of_remote_repository> is <existing/original_repository_name>, and of course, new and existing repository names can be the same. Also, there is a little more work to point local git to new remote as origin, and remove stale remote repository pointers. For help on that topic please refer git tutorial. Repo FAQHow do I contact someone about this facility?Email repo-support@eecs. How do I update this faq?See How do I contact someone about this facility? What is the usage policy of EECS Repo?EECS Repo is hosted at the University of California, Berkeley, so the UC Berkeley Computer Use Policy applies. Use your common sense. Don't upload movies and music unless you have a license. Be very careful about personally identifiable information. This site operates under the Privacy Statement for University of California, Berkeley Websites, which may change at any time. How do collaborators who do not have UC Berkeley EECS LDAP accounts get access to a repository?Someone with a UC Berkeley EECS LDAP account must create a repository and then add offsite users. When an external user account is created, email is sent to that address. The person who creates the repository should email the URL of the repository to the external user. External usernames are their email address, for example External account users may change their password by going to https://repo.eecs.berkeley.edu/ and clicking on "Change your password" button and then logging in. External account users may recover their username or password by going following the password reset instructions. How do I transfer control of a repository to someone else?Go to Access/Notification, select your repository and add/remove users in 'Assigned users' section. Set their permission R or RW. If you need to add user to manage your space ( all repositories), go to 'Space permission' and assign user with administrator permission. Only EECS account holders are permitted in this section. Allow others to administer non-personal spaces?Once you are logged in to the https://repo.eecs.berkeley.edu gui, select the space. Then use the link "space permission" on the left. Browse or type the user name you want to provide permission to, select administrator role and click the "assign" button. Right column shows current users with space administrative rights. Only Eecs account holders can be set as admins. Why isn't Mercurial/scm package de jour available?Rather than starting from scratch, we based the site on iF.SVNAdmin, which supports only Subversion. The iF.SVNAdmin package uses Subversion's path-based authorization. Kostadin Ilov extended iF.SVNAdmin to support Git. This user interface is horrible!The user interface is not perfect, but once you get over a few hurdles, it becomes easier. The source code for the UI will be made available and we are happy to collaborate with people who want to make changes. To get access to the source code, send email to Kostadin (kilov at eecs). The repository is at Why can't I create a Non-Personal Space?Non-personal space names are shared amongst all EECS Repo users. We want to avoid clutter and also manage the names so that spaces get appropriate names. When deciding whether to request a non-personal space consider the following:
To request the creation of a new non-personal space, please use the Space Selector web page to request a new space. Creating SpacesWhen a new space is requested, email is sent to the group of EECS Repo administrators. In general, our policy is to create new spaces when they are requested, providing that there does not seem to be a conflict or ambiguity in the naming. If the requestor and a EECS Repo administrator are members of the same center or group, then that EECS Repo administrator should create the group. Otherwise, either Christopher or Kostadin may typically create the project, though anyone is welcome to do so. Below is the procedure for creating a space that is to be loosely followed by EECS Repo administrators:
Why doesn't code.berkeley.edu meet our needs?It appears that https://code.berkeley.edu was replaced with the Berkeley GitHub. This sort of discontinuation of services points out the need for having a local, long-lived repository, as opposed to depending on things like https://code.berkeley.edu or Google code, both of which have been discontinued. At one time https://code.berkeley.edu/ had no way to share repositories with non-Calnet users. This probably changed in 2012: CalNet Guest Status as of May 23, 2012 Why not use github or another commercial solution?If GitHub meets your needs, then by all means, use it. Here are some things to consider:
Who pays for space?Right now, the disk space is free. As disk space usage increases, we may address usage by having repositories located in project space. Who created the EECS Repo?Kostadin Ilov did most of the heavy work. The EECS Research-Blue Team provided feedback: Jeff Anderson-Lee, Christopher Brooks, Ubirata Chaves Coelho, Aaron Culich, Eric Fraser, Jon Kuroda, Brian Shiratsuki, Keith Sklower and Mary Stewart. The project stemmed from conversations with Professor James O'Brien. Problems and their Solutionssvn: access to xxx forbiddenProblem: bash-3.2$ svn co https://repo.eecs.berkeley.edu/svn/projects/projectName/repository svn: access to 'https://repo.eecs.berkeley.edu/svn/projects/projectName/Repository' forbidden Possible solutions:
svn co --username yourUserName https://repo.eecs.berkeley.edu/svn/projects/svn/projectName/repository where git push returns "Your configuration specifies to merge with the ref 'master' from the remote, but no such ref was fetched"The very first git push per each repository MUST be in format: $git push -u origin master after that any subsequently pushes can be simple: git clone returns "error: Protocol https not supported or disabled in libcurl while accessing"http://chess.eecs.berkeley.edu/ptexternal/wiki/Main/Git#Curl git clone returns "Gtk-WARNING **: cannot open display"See http://chess.eecs.berkeley.edu/ptexternal/wiki/Main/Git#Gtk-WARNING git clone returns "error: The requested URL returned error: 401 while accessing"[cxh@joulecentos ~]$ cat /etc/redhat-release Red Hat Enterprise Linux Workstation release 6.2 (Santiago) [cxh@joulecentos ~]$ git --version git version 1.7.1 [cxh@joulecentos ~]$ git clone https://repo.eecs.berkeley.edu/git/projects/embedded/gitTest.git Initialized empty Git repository in /home/cxh/gitTest/.git/ error: The requested URL returned error: 401 while accessing https://repo.eecs.berkeley.edu/git/projects/embedded/gitTest.git/info/refs fatal: HTTP request failed [cxh@joulecentos ~]$ Try adding the username to the url: [cxh@joulecentos ~]$ git clone https://cxh@repo.eecs.berkeley.edu/git/projects/embedded/gitTest.git Initialized empty Git repository in /home/cxh/gitTest/.git/ Password: warning: You appear to have cloned an empty repository. [cxh@joulecentos ~]$ git clone returns "error: server certificate verification failed."The repository server (like many other departmental servers) uses an InCommon derived certificate and some machines do not include the InCommon root certificate in the default ca-certificates file. You may be able to add the missing certificate to your ca-certificates file, as follows: wget https://repo.eecs.berkeley.edu/svnadmin/data/InCommonServerCA.pem cat /etc/ssl/certs/ca-certificates.crt InCommonServerCA.pem > ca-certs2.crt sudo cp /etc/ssl/certs/ca-certificates.crt /etc/ssl/certs/ca-certificates.crt.bak sudo cp ca-certs2.crt /etc/ssl/certs/ca-certificates.crt If the above does not work for you, then try these steps below: 1. Go to https://iris.eecs.berkeley.edu/news/6499-new-ssl-certificate-for-wireless and download the Root and Intermediate certificates
2. Create a director and copy the two certificate files into it sudo mkdir –p /usr/share/ca-certificates/repo sudo cp <location_of_downloaded_certs>/*.crt /usr/share/ca-certificates/repo/ 3. Edit repo/AddTrustExternalCARoot.crt repo/COMODOHigh-AssuranceSecureServerCA.crt repo/InCommonServerCA.pem 4. Run sudo update-ca-certificates If you are using anything but Ubuntu, the goal is to have the two certificates included in /etc/ssl/certs/ca-certificates.crt or wherever they are visible by CURL ( which is what GIT is using for https access). MAC users Once you downloaded the two certificates from IRIS site, you have to append them to curl-ca-bundle.crt file that CURL is using. For Mac OS X 10.6, the default location is /usr/share/curl/curl-ca-bundle.crt: sudo -i cat /usr/share/curl/curl-ca-bundle.crt first_downloaded_cert second_downloaded cert > /usr/share/curl/curl-ca-bundle_new.crt cp /usr/share/curl/curl-ca-bundle.crt /usr/share/curl/curl-ca-bundle_old.crt mv /usr/share/curl/curl-ca-bundle_new.crt /usr/share/curl/curl-ca-bundle.crt MacOS X 10.7 and newer have different approach with curl certificates and utilize Key Chain Access instead. To Do ItemsBetter explanation for Space concept and better communication of that idea to the users11/9/12 Implementation or explanation why not for second ( or more) SSH keys per user11/9/12 Documenting the correct steps to enter the SSL certs for https on client machines11/9/12. See git clone returns "error: server certificate verification failed." Set up a way to use svn-hooks, or at least a way to send email on commits.11/21/12. This is under development. Until there is an automated method, to have a hook added, send email to the EECS Repo support team. |