/

The Ultimate Guide to HTTP Requests with Curl

The Ultimate Guide to HTTP Requests with Curl

Curl is a command-line tool that allows you to transfer data across the network. It supports a wide range of protocols, including HTTP, HTTPS, FTP, FTPS, SFTP, IMAP, SMTP, POP3, and more. This tool is extremely useful for debugging network requests and is considered a programmer’s best friend.

Installing Curl

Curl is a universal tool that can be installed on Linux, Mac, and Windows. You can refer to the official installation guide to install it on your system.

Performing HTTP Requests

Let’s dive into some of the most common HTTP requests and operations you’ll likely want to perform using curl.

Perform an HTTP GET Request

To perform an HTTP GET request using curl, simply use the following command:

1
curl https://flaviocopes.com/

Get the HTTP Response Headers

By default, curl hides the response headers in its output. To display the headers, use the -i option:

1
curl -i https://flaviocopes.com/

Only Get the HTTP Response Headers

If you only want to retrieve the headers and not the response body, you can use the -I option:

1
curl -I https://flaviocopes.com/

Perform an HTTP POST Request

To perform an HTTP POST request, you can use the -X POST option. By default, curl uses the GET method. Here’s an example:

1
curl -X POST https://flaviocopes.com/

Perform an HTTP POST Request Sending JSON

If you want to send JSON data in your POST request, you need to set the Content-Type header explicitly. Here’s how you can do it:

1
curl -d '{"option": "value", "something": "anothervalue"}' -H "Content-Type: application/json" -X POST https://flaviocopes.com/

Perform an HTTP PUT Request

Similar to POST requests, you can perform an HTTP PUT request using the -X PUT option:

1
curl -X PUT https://flaviocopes.com/

Follow a Redirect

If you encounter a redirect response (e.g., a 301 error with a Location header), you can automatically follow the redirect by using the -L option:

1
curl -L http://flaviocopes.com/

Store the Response to a File

To save the response to a file, use the -o option followed by the desired filename:

1
curl -o file.html https://flaviocopes.com/

If you want to save the file with its original name from the server, use the -O option:

1
curl -O https://flaviocopes.com/index.html

Using HTTP Authentication

If a resource requires Basic HTTP Authentication, you can include the user and password in the request using the -u option:

1
curl -u user:pass https://flaviocopes.com/

Set a Different User Agent

Curl sends the curl/<version> user agent by default. However, you can specify a different user agent using the --user-agent option:

1
curl --user-agent "my-user-agent" https://flaviocopes.com

Inspecting All the Details of the Request and Response

To output all the details of the request and response, use the --verbose option:

1
curl --verbose -I https://flaviocopes.com/

This will provide you with a comprehensive overview of the request and response, including TCP connections, TLS certificates, headers, and more.

Copying a Browser Network Request to Curl

If you’re using Chrome Developer Tools and want to copy a network request as a curl command, you can do so by right-clicking on the request and selecting “Copy as curl”:

1
curl 'https://github.com/curl/curl' -H 'Connection: keep-alive' -H 'Pragma: no-cache' -H 'Cache-Control: no-cache' -H 'Upgrade-Insecure-Requests: 1' -H 'DNT: 1' -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36' -H 'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8' -H 'Referer: https://www.google.it/' -H 'Accept-Encoding: gzip, deflate, br' -H 'Accept-Language: en-US,en;q=0.9,it;q=0.8' -H 'Cookie: _octo=GH1.1.933116459.1507545550; _ga=GA1.2.643383860.1507545550; tz=Europe%2FRome; user_session=XXXXX; __Host-user_session_same_site=YYYYYY; dotcom_user=flaviocopes; logged_in=yes; has_recent_activity=1; _gh_sess=ZZZZZZ' --compressed

Conclusion

Curl is an amazing tool for working with HTTP requests. With its support for various protocols and its ability to debug network requests, it’s an invaluable asset for developers. Make sure to explore all the possibilities and options curl offers to optimize your workflow.

tags: [“curl”, “HTTP requests”, “network requests”, “debugging”, “command-line”]