Gitlab API 获取所有项目信息

为了所有Gitlab 服务器上所有projectName、path、nameSpace、branchs等信息，需要用到Gitlab API.

首先需要配置PRIVATE-TOKEN，在Gitlab Server，Account-setting中配置TOKEN

开始API操作

浏览具有访问权限的项目：

curl --header "PRIVATE-TOKEN: EeVPPydkxJuxhaabbbgb1" https://git.kavenran.com/api/v4/projects

返回的JSON数据如(官方示例)：

[
  {
    "id": 4,
    "description": null,
    "default_branch": "master",
    "visibility": "private",
    "ssh_url_to_repo": "git@example.com:diaspora/diaspora-client.git",
    "http_url_to_repo": "http://example.com/diaspora/diaspora-client.git",
    "web_url": "http://example.com/diaspora/diaspora-client",
    "tag_list": [
      "example",
      "disapora client"
    ],
    "owner": {
      "id": 3,
      "name": "Diaspora",
      "created_at": "2013-09-30T13:46:02Z"
    },
    "name": "Diaspora Client",
    "name_with_namespace": "Diaspora / Diaspora Client",
    "path": "diaspora-client",
    "path_with_namespace": "diaspora/diaspora-client",
    "issues_enabled": true,
    "open_issues_count": 1,
    "merge_requests_enabled": true,
    "jobs_enabled": true,
    "wiki_enabled": true,
    "snippets_enabled": false,
    "container_registry_enabled": false,
    "created_at": "2013-09-30T13:46:02Z",
    "last_activity_at": "2013-09-30T13:46:02Z",
    "creator_id": 3,
    "namespace": {
      "id": 3,
      "name": "Diaspora",
      "path": "diaspora",
      "kind": "group",
      "full_path": "diaspora"
    },
    "import_status": "none",
    "archived": false,
    "avatar_url": "http://example.com/uploads/project/avatar/4/uploads/avatar.png",
    "shared_runners_enabled": true,
    "forks_count": 0,
    "star_count": 0,
    "runners_token": "b8547b1dc37721d05889db52fa2f02",
    "public_jobs": true,
    "shared_with_groups": [],
    "only_allow_merge_if_pipeline_succeeds": false,
    "only_allow_merge_if_all_discussions_are_resolved": false,
    "request_access_enabled": false,
    "approvals_before_merge": 0,
    "statistics": {
      "commit_count": 37,
      "storage_size": 1038090,
      "repository_size": 1038090,
      "lfs_objects_size": 0,
      "job_artifacts_size": 0
    }
  },
  ...
]

如果觉得返回数据太多，可以使用简化版本：
curl --header "PRIVATE-TOKEN: EeVPPydkxJuxhqnaaagb1" https://git.kavenran.com/api/v4/projects?simple=true
返回数据格式如：

[
    {
        "id": 218,
        "http_url_to_repo": "https://git.kavenran.com/Stars/ONE.git",
        "web_url": "https://git.kavenran.com/Stars/ONE",
        "name": "ONE",
        "name_with_namespace": "Stars / ONE",
        "path": "ONE",
        "path_with_namespace": "Stars/ONE"
    },
    ...
]

特别注意，返回的数据是分页的，每页默认20条，如果项目较多需要使用分页参数。
curl --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/8/issues/8/notes?per_page=50

如何知道有多少页呢？

curl --head --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/8/issues/8/notes?per_page=50

HTTP/1.1 200 OK
Cache-Control: no-cache
Content-Length: 1103
Content-Type: application/json
Date: Mon, 18 Jan 2016 09:43:18 GMT
Link: <https://gitlab.example.com/api/v4/projects/8/issues/8/notes?page=1&per_page=3>; rel="prev", <https://gitlab.example.com/api/v4/projects/8/issues/8/notes?page=3&per_page=3>; rel="next", <https://gitlab.example.com/api/v4/projects/8/issues/8/notes?page=1&per_page=3>; rel="first", <https://gitlab.example.com/api/v4/projects/8/issues/8/notes?page=3&per_page=3>; rel="last"
Status: 200 OK
Vary: Origin
X-Next-Page: 2
X-Page: 1
X-Per-Page: 50
X-Prev-Page:
X-Request-Id: 732ad4ee-9870-4866-a199-a9db0cde3c86
X-Runtime: 0.108688
X-Total: 140
X-Total-Pages: 3

X-Total-Pages: 3 就是总的页数，访问下一页：

curl --head --header "PRIVATE-TOKEN: 9koXpg98eAheJpvBs5tK" https://gitlab.example.com/api/v4/projects/8/issues/8/notes?per_page=50&pages=2

其它API接口及参数请查阅：
https://docs.gitlab.com/ee/api/README.html

如果本文到这里就结束了，那只能是一篇坑文，下面shell库实现了获取服务器所有项目信息，并组装为以下格式的数据：

|| groupname || projectname || https_url || default_branch ||

function gitlabRepositry(){

	loginfo $BASH_SOURCE "开始获取仓库信息-----"

	[[ -e .gitlab_result ]] && rm -f .gitlab_result

	total_pages=`curl --head --header "PRIVATE-TOKEN:$TOKEN" "$GITSERVER?per_page=50" | grep '^X-Total-Pages' | sed 's/X-Total-Pages: //g' | sed 's/\r//g'`

	for (( p=1;p<$total_pages;p++ )) {

		curl --header "PRIVATE-TOKEN: $TOKEN" "$GITSERVER?per_page=50&page=$p" > .gitlab_projects

		cat .gitlab_projects | ./libs/JSON.sh -b | awk '$1 ~ /http_url_to_repo/ {print $2}' > .temp_http_url_to_repo

		cat .gitlab_projects | ./libs/JSON.sh -b | awk '$1 ~ /name_with_namespace/ {print $2"\t"$4}' > .temp_path_with_namespace


		cat .gitlab_projects | ./libs/JSON.sh -b | awk '$1 ~ /default_branch/ {print $2}' | awk -F "/" '{print $1"\t"$2}'> .temp_default_branch

		paste .temp_path_with_namespace .temp_http_url_to_repo .temp_default_branch >> .gitlab_result
	}

	#替换所有的引号
	replacefile .gitlab_result

	#git地址添加用户名
	addUser .gitlab_result

	#rm -f .temp_*

	loginfo $BASH_SOURCE "获取仓库信息完成"
}

function replacefile(){
	if ([ -n "$1" ] && [ -e "$1" ])
        then
		sed -i 's/\"//g' $1
 	fi
}

function addUser(){

	if ([ -n "$1" ] && [ -e "$1" ])
        then
		sed -i 's/https:\/\//https:\/\/'"$USER_NAME"'@/g' $1
 	fi
}

获取的数据在.gitlab_result中：

SkyOne     DESK     https://kaven@git.kavenran.com/SkyOne/DESK.git master
SkyOne    XBING  https://kaven@git.kavenran.com/SkyOne/XBING.git      master
SkyTwo   WANL     https://kaven@git.kavenran.com/SkyTwo/WANL.git        master