cURL简介
cURL是一款使用C语言开发的,利用URL语法在命令行下工作的文件传输工具,1997年首次发行。它支持文件上传和下载,所以是综合传输工具。
命令使用:
除了使用它来进行HTTP试的上传、下载,一般我们还用它来检测网址URL的状态码,来判断网站状态。
常用判断,使用-I只打印信息网站信息头:
[root@test ~]# curl -I http://www.baidu.com HTTP/1.1 200 OK Server: bfe/1.0.8.18 Date: Tue, 20 Dec 2016 09:15:45 GMT Content-Type: text/html Content-Length: 277 Last-Modified: Mon, 13 Jun 2016 02:50:44 GMT Connection: Keep-Alive ETag: "575e1f84-115" Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform Pragma: no-cache Accept-Ranges: bytes
可以看到,目标网站返回HTTP/1.1 200 OK等信息,表示目标网站已正常处理连接请求。
有时候我们可能需要制作脚本,来判断网站状态,只需要状态码,不需要显示其它信息时,我们可以使用-w来指定要显示的信息:
[root@test ~]# curl -I -o /dev/null -s -w %{http_code}"\n" http://www.baidu.com 200
-o /dev/null代表将输出到系统空洞,即不显示;-s表示安静模式;-w %{http_code}表示显示http状态码,后面的”\n”表示在后面插入换行符,其得出结果后不会自动换行。
这样一来可以判断大部分网站URL的状态是否正常,但有几种情况:
- 网站具有重定向,其会返回301、302代码:
[root@test ~]# curl -I -o /dev/null -s -w %{http_code}"\n" http://www.cn-markets.com/ 301 [root@test ~]# curl -I http://www.cn-markets.com/ HTTP/1.1 301 Moved Permanently Server: nginx Date: Tue, 20 Dec 2016 10:43:50 GMT Content-Type: text/html Content-Length: 178 Connection: keep-alive Location: http://www.markets.com.cn/ Set-Cookie: visid_incap_686916=BpHl+6DCQl+GU0ay7hijxGULWVgAAAAAQUIPAAAAAABDXY1oomo2ONQywWLKFI6d; expires=Wed, 20 Dec 2017 09:45:15 GMT; path=/; Domain=.cn-markets.com Set-Cookie: incap_ses_560_686916=zwQLWVMpGCNZzFMpFIXFB2ULWVgAAAAAZOoRdMvCLqtO84+WrN8RmA==; path=/; Domain=.cn-markets.com X-Iinfo: 7-63385929-63386626 SNNN RT(1482230505033 124500) q(0 0 0 -1) r(0 0) U5 X-CDN: Incapsula Powered-By-ChinaCache: MISS from CHN-JA-a-3S6 Cache-Control: no-cache,no-store,must-revalidate Pragma: no-cache Powered-By-ChinaCache: MISS from CHN-NG-3-3WG
如果要让其能判断跳转后的URL,须加入-L选项:
[root@test ~]# curl -I -L -o /dev/null -s -w %{http_code}"\n" http://www.cn-markets.com/ 200
- 有些网站限制了访问方式,否则会报404、405错误:
[root@test ~]# curl -I -L -o /dev/null -s -w %{http_code}"\n" http://www.ym206.com 404 [root@nagios ~]# curl -I http://www.ym206.com HTTP/1.1 404 Not Found Content-Length: 390 Content-Type: text/html Server: Microsoft-IIS/7.5 X-Powered-By: ASP.NET Date: Tue, 20 Dec 2016 11:31:57 GMT [root@test ~]# curl -I -L -o /dev/null -s -w %{http_code}"\n" http://www.24k.com/ 405 [root@test ~]# curl -I http://www.24k.com/ HTTP/1.1 405 Method Not Allowed Server: nginx/1.9.0 Date: Tue, 20 Dec 2016 11:04:57 GMT Content-Type: text/html;charset=UTF-8 Connection: keep-alive Allow: GET Set-Cookie: JSESSIONID=A41E85B56973B9A7DADFAE8544864BFA; Path=/; HttpOnly
这种情况需要添加-X参数指定以GET方式获取页面:
[root@test ~]# curl -X GET -L -o /dev/null -s -w %{http_code}"\n" http://www.ym206.com 200 [root@test ~]# curl -X GET -L -o /dev/null -s -w %{http_code}"\n" http://www.24k.com/ 200
- 同时,有的网站进一步指定了须提交User-Agent,否则同样会报403错误:
[root@test ~]# curl -I -L -o /dev/null -s -w %{http_code}"\n" http://www.qixin.com/company/9ae5e956-14cd-4674-95bf-d7224b8dd057 403 [root@test ~]# curl -I http://www.qixin.com/company/9ae5e956-14cd-4674-95bf-d7224b8dd057 HTTP/1.1 403 Forbidden Date: Tue, 20 Dec 2016 11:37:08 GMT Content-Type: text/html Content-Length: 168 Connection: keep-alive Set-Cookie: aliyungf_tc=AQAAAJ+0/Tr4KQYAg6WBt6fP40FQMqQn; Path=/; HttpOnly Server: nginx/1.4.1
这种情况,加上-A参数指定代理字符串即可:
[root@nagios ~]# curl -X GET -L -A "Chrome" -o /dev/null -s -w %{http_code}"\n" http://www.qixin.com/company/9ae5e956-14cd-4674-95bf-d7224b8dd057 200
- 更有甚者,还须指定提供浏览器语言环境,否则会报500错误:
[root@test ~]# curl -X GET -L -A "Chrome" -o /dev/null -s -w %{http_code}"\n" http://www.fx77.hk 500 [root@test ~]# curl -I http://www.fx77.hk HTTP/1.1 500 Internal Server Error Content-Length: 1141 Content-Type: text/html Server: Microsoft-IIS/8.0 Date: Tue, 20 Dec 2016 11:07:30 GMT
这种情况,还需要添加-H来添加额外header信息,指定接受语言方可成功获取:
[root@test ~]# curl -X GET -L -A "Chrome" -H "Accept-Language:zh-CN,zh;q=0.8" -o /dev/null -s -w %{http_code}"\n" http://www.fx77.hk 200
- 除此之外,有的网站还要求必须接受Cookies,否则还会报302错误:
[root@test ~]# curl -X GET -L -A "Chrome" -H "Accept-Language:zh-CN,zh;q=0.8" -o /dev/null -s -w %{http_code}"\n" http://www.88bank.com/ 302
使用-b指定cookies字符串或文件即可:
[root@test ~]# curl -X GET -L -A "Chrome" -H "Accept-Language:zh-CN,zh;q=0.8" -b cookiestrings -o /dev/null -s -w %{http_code}"\n" http://www.nioc.ir/ 200
以上情况还需根据实际遇到的现象多做分析加以判断。
同时curl还有一个参数也较为实用,-m可以指定连接目标网站的超时时间,单位为秒:
[root@test ~]# curl -X GET -L -A "Chrome" -H "Accept-Language:zh-CN,zh;q=0.8" -m 7 -b cookiestrings -o /dev/null -s -w %{http_code}"\n" http://www.nosuchdomain.com 000
这样就不必因为访问不到目标服务器而浪费太多时间。
I loved as much as you will receive carried out right here.
The sketch is attractive, your authored material stylish.
nonetheless, you command get got an impatience over that you wish be delivering
the following. unwell unquestionably come more formerly again since exactly the same nearly
very often inside case you shield this hike.