Ways to Retrieve Body Content from an HTTPS URL Using CURL

Is there a way to modify the following code to retrieve the body content of a URL using PHP's CURL, specifically for HTTP and not HTTPS? I need to access the data returned, not just the header information. Any suggestions or guidance would be greatly appreciated.

After running a test, I found that the result includes a content-length value but I'm unsure how to extract it. Can anyone provide insight on how to do this?

Thank you, Stephen

Errors: 0

string(1457) "HTTP/1.1 200 OK Date: Sat, 01 Aug 2009 06:32:11 GMT Server: Apache/1.3.41 (Darwin) PHP/5.2.4 mod_ssl/2.8.31 OpenSSL/0.9.7l Cache-Control: max-age=60 Expires: Sat, 01 Aug 2009 06:33:11 GMT Last-Modified: Thu, 23 Nov 2006 17:44:53 GMT ETag: "97d620-44b-4565de15" Accept-Ranges: bytes Content-Length: 1099 Connection: close Content-Type: text/html "

<?php

$curl_handle=curl_init();

$username = "";
$password = "";

$fullurl = "http://www.queensberry.com";
   $ch = curl_init();
   curl_setopt($ch, CURLOPT_HEADER, 1);
   curl_setopt($ch, CURLOPT_VERBOSE, 1);
   curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
   curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
   curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
   curl_setopt($ch, CURLOPT_FAILONERROR, 0);
   curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
   curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
   curl_setopt($ch, CURLOPT_URL, $fullurl);

   $returned = curl_exec($ch);

   curl_close ($ch);
   var_dump($returned);


?>

Answer №1

Providing a solution: Please attempt the following steps while keeping the rest of the code unchanged...

$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_FAILONERROR, 0);
// curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$password");
curl_setopt($ch, CURLOPT_URL, $fullurl);

$returned = curl_exec($ch);

curl_close ($ch);
var_dump($returned);

Adjusting CURLOPT_HEADER to 0 ensures that only the content of the page is retrieved.

Answer №2

When I need to determine the length of a header, I utilize curl's getinfo function. Then, I proceed to extract a substring from the response:

$headers = curl_getinfo($ch);
$start_position = $headers['header_size'];
$body_content = substr($response, $start_position, strlen($response) - $start_position);

Answer №3

Does it make sense for $fullurl to be set to "" instead?

After updating $fullurl as instructed, and executing the script, a var_dump outputted the "coming soon" page.

Answer №4

If you require the header and cannot set CURLOPT_HEADER to 0, you can locate the beginning of the body by searching for a double empty line (two CRLF). Refer to the specification at: http://www.w3.org/Protocols/rfc2616/rfc2616-sec6.html

The following code snippet should accomplish this:

    $data = curl_exec($ch);
    $start = strpos($data, "\r\n\r\n") + 4;
    $body = substr($data, $start, strlen($data) - $start);

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Basic emailing list linked to database

Currently, I am working on developing a mailing list feature that will save email addresses into an existing MySQL database after the user enters their first name, last name, and email and clicks on a submit button. Once this process is completed, I want a ...

Finding it challenging to retrieve the dynamically generated data from the PHP-MYSQL combination through jQuery AJAX

The question's title doesn't provide much information. Let me explain what I'm trying to accomplish here. Using jQuery, I have an HTML form that sends an AJAX request to a PHP file. The PHP file successfully returns the required data back to ...

Manipulating multidimensional arrays in PHP and converting them to JSON format

I have a MySQL table with the following fields: id | building | title | parent Now, I am looking to implement a timeline calendar using fullcalendar.io. This is the structure I aim to create: { id: '1', building: '460 Bryant', title ...

Discover the integer value through .htaccess redirection

I have a profile system in place on my website. I have set up the redirect from example.com/player/?id=1 to example.com/player/1. How can I ensure that once they are redirected, I can identify that the user ID is actually 1 and not face a 404 error? .htac ...

Incorporating a WHERE clause into the COUNT query

Having some difficulty incorporating a where clause into a COUNT query. The following code successfully retrieves the total count of the table: $search = $_POST['search_text']; SELECT COUNT(*) FROM stories') ->fetchColumn(); However ...

How can I retrieve query parameters passed from an Angular application in PHP?

I'm trying to figure out how to retrieve data from query parameters sent by an Angular application in PHP. Unfortunately, I don't have much experience with PHP. Is there anyone willing to lend a hand? ...

Difficulty in transferring a PHP variable to an AJAX file through json_encode

Currently working on a multi-phase form where the user progresses through each phase by completing different sections. In the first phase, the user fills out a form that is then submitted to the backend for validation and data computation. Once validated, ...

optimal method for managing lists that are both stationary and able to grow

Here is the structure of my database: 'Article' title text user_id language_id 'User' id language_id 'language' id language name Is it best practice to store a list of languages in a MySQL database? I am a desig ...

Executing an authenticated Guzzle POST request

Just starting out with guzzle and I need help on how to make a Guzzle call for this specific request. curl -X POST -u username:password -H "Content-Type: application/json" https://xyz/api/v1/accounts.json -d '{"user":{"username":"test","password":"xy ...

Can setting PHP error_reporting(0) impact both error logging and display, or just how errors are shown?

Can using error_reporting(0); impact error logging to a file, or does it simply hide errors from being displayed on screen? Appreciate your input. ...

Encountered an issue when attempting to generate a fresh Laravel project with Composer

I'm a beginner with Laravel and I've been attempting to create a fresh project using composer, but I keep encountering an error that I can't seem to resolve. Whenever I try to create a new project in the /htdocs directory using the command ...

Introducing Laravel 6's Hidden Gems: Unleash the Power of @push

Hey everyone, I'm a newcomer to the world of Laravel and currently using Laravel 6.0 I've encountered an issue with my javascript code that utilizes @push. Strangely enough, the script only functions properly when I manually insert the code into ...

What is the best way to track the loading progress of an AJAX page in WordPress?

On my WordPress blog, I utilize a plugin known as Advanced Ajax Page Loader. This handy tool loads the next page or post through AJAX and then places it in a specific div on my site. However, I am now interested in incorporating a progress bar to show the ...

The combination of Mysql's unix_timestamp function with date_add is producing inaccurate timestamps

I am experiencing an issue where I cannot get the correct unix_timestamp after using date_add on my database field. Instead of getting the unix_timestamp after the date_add, it is returning the unix_timestamp of the original value. Below is the query I am ...

Achieving the functionality of keeping the search query box and alphabetical search options visible on the page even after performing a search and displaying the results can be done by

I have set up a PHP page with search query field and alphabetical search options. When submitting the query, the results are displayed on the same page; however, I would like the results to show in the same location as the alphabetical search. Currently, ...

The Ajax data is appearing in the console, however, it is not being recognized as a valid PHP value

Hey there, I have a question about utilizing Ajax. I've searched for similar issues but can't seem to find anything that matches my problem. Below is the code in question: $(document).ready(function() { $("#nume").blur(function() { numeform = ...

Is it possible to pass a PHP array to JavaScript without using Ajax?

Currently, I have a JavaScript function that utilizes Ajax to fetch an array of data from PHP and dynamically populates a dropdown menu. Everything is functioning as expected. However, I am beginning to feel that using Ajax for this task might be a bit ex ...

Are HTML files and PHP generated files cached differently by web browsers?

My current setup involves Nginx as a web server and Firefox to check response headers. I added two files on the server - test.html and test.php, both containing the same content. In the Nginx config file, I have set the expires directive to 30d in the serv ...

"Communicate through real-time chat applications by leveraging node.js, socket.io,

Looking to create a chat system using nodejs and socket.io, I have a working prototype but the question remains: how should I handle storing chat messages in the database? It doesn't seem practical to store every message as soon as a user presses ent ...

What is the best way to create a unique collection that highlights the most frequently occurring data points?

I have a table setup as shown below: https://i.stack.imgur.com/erxnt.png This particular table is designated as favourite_products and holds product ids that users have marked as favorites. My goal is to retrieve a list of the most frequently favorited ...