The Ultimate Guide to HTTP Requests with Curl
Curl is a command-line tool that allows you to transfer data across the network. It supports a wide range of protocols, including HTTP, HTTPS, FTP, FTPS, SFTP, IMAP, SMTP, POP3, and more. This tool is extremely useful for debugging network requests and is considered a programmer’s best friend.
Installing Curl
Curl is a universal tool that can be installed on Linux, Mac, and Windows. You can refer to the official installation guide to install it on your system.
Performing HTTP Requests
Let’s dive into some of the most common HTTP requests and operations you’ll likely want to perform using curl.
Perform an HTTP GET Request
To perform an HTTP GET request using curl, simply use the following command:
1 | curl https://flaviocopes.com/ |
Get the HTTP Response Headers
By default, curl hides the response headers in its output. To display the headers, use the -i option:
1 | curl -i https://flaviocopes.com/ |
Only Get the HTTP Response Headers
If you only want to retrieve the headers and not the response body, you can use the -I option:
1 | curl -I https://flaviocopes.com/ |
Perform an HTTP POST Request
To perform an HTTP POST request, you can use the -X POST option. By default, curl uses the GET method. Here’s an example:
1 | curl -X POST https://flaviocopes.com/ |
Perform an HTTP POST Request Sending JSON
If you want to send JSON data in your POST request, you need to set the Content-Type header explicitly. Here’s how you can do it:
1 | curl -d '{"option": "value", "something": "anothervalue"}' -H "Content-Type: application/json" -X POST https://flaviocopes.com/ |
Perform an HTTP PUT Request
Similar to POST requests, you can perform an HTTP PUT request using the -X PUT option:
1 | curl -X PUT https://flaviocopes.com/ |
Follow a Redirect
If you encounter a redirect response (e.g., a 301 error with a Location header), you can automatically follow the redirect by using the -L option:
1 | curl -L http://flaviocopes.com/ |
Store the Response to a File
To save the response to a file, use the -o option followed by the desired filename:
1 | curl -o file.html https://flaviocopes.com/ |
If you want to save the file with its original name from the server, use the -O option:
1 | curl -O https://flaviocopes.com/index.html |
Using HTTP Authentication
If a resource requires Basic HTTP Authentication, you can include the user and password in the request using the -u option:
1 | curl -u user:pass https://flaviocopes.com/ |
Set a Different User Agent
Curl sends the curl/<version> user agent by default. However, you can specify a different user agent using the --user-agent option:
1 | curl --user-agent "my-user-agent" https://flaviocopes.com |
Inspecting All the Details of the Request and Response
To output all the details of the request and response, use the --verbose option:
1 | curl --verbose -I https://flaviocopes.com/ |
This will provide you with a comprehensive overview of the request and response, including TCP connections, TLS certificates, headers, and more.
Copying a Browser Network Request to Curl
If you’re using Chrome Developer Tools and want to copy a network request as a curl command, you can do so by right-clicking on the request and selecting “Copy as curl”:
1 | curl 'https://github.com/curl/curl' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'DNT: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: https://www.google.it/' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9,it;q=0.8' -H 'Cookie: _octo=GH1.1.933116459.1507545550; _ga=GA1.2.643383860.1507545550; tz=Europe%2FRome; user_session=XXXXX; __Host-user_session_same_site=YYYYYY; dotcom_user=flaviocopes; logged_in=yes; has_recent_activity=1; _gh_sess=ZZZZZZ' --compressed |
Conclusion
Curl is an amazing tool for working with HTTP requests. With its support for various protocols and its ability to debug network requests, it’s an invaluable asset for developers. Make sure to explore all the possibilities and options curl offers to optimize your workflow.
tags: [“curl”, “HTTP requests”, “network requests”, “debugging”, “command-line”]