I need to know how to output a list of URLs from a webpage using Selenium chrome driver in C# to the console. Can you help

I am trying to achieve the task of printing all URLs in a specific location on a website to the console. While I have managed to extract the text of all the links, I am struggling with obtaining the actual URLs. Since I am new to coding, any assistance would be greatly appreciated. I have been advised to use a different web driver, however, for my current project, I prefer to stick with Selenium.

using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Support;
using System;
using System.Collections.Generic;
using System.IO;
using System.Text;
using OpenQA.Selenium.Support.UI;

namespace Test_Scraper_1
{
    class Program
    {
        static void Main(string[] args)
        {
            //Initialize chrome driver
            using (var driver = new ChromeDriver())
            {
                driver.Navigate().GoToUrl("https://www.tfrrs.org/");

                //find elements
                var Search_Field = driver.FindElementByXPath(@"/html/body/div[3]/div/div/div[4]/div/div[2]/form/div[1]/input");
                var Search_Button = driver.FindElementByXPath(@"/html/body/div[3]/div/div/div[4]/div/div[2]/form/div[4]/button");

                var Count = 1;

                Search_Field.SendKeys("Ashley Smith");
                Search_Button.Click();

                var titles = driver.FindElementsByClassName("allRows");
                foreach (var allRows in titles)
                {
                    Console.WriteLine(allRows.Text + Count++);
                }

                Console.ReadLine();
            }
        }
    }
}

Answer №1

Your allRows contains a tr element as shown below.

<tr class="filtered allRows ">
    <td id="col0">
        <a href="//www.tfrrs.org/athletes/3721395/Youngstown_St/Ashley__Smith_.html">Ashley  Smith </a>
    </td>

    <td id="col1">
        <a href="//www.tfrrs.org/teams/xc/OH_college_f_Youngstown_St.html">Youngstown St. (F)</a>
    </td>
</tr>

To retrieve the href attribute of the a element, you can use the following code snippet to get the link from the first column:

var column0 = allRows.FindElement(By.Id("col0"));
var aElement = column0.FindElement(By.TagName("a"));
var link = aElement.GetAttribute("href");

Answer №2

When iterating through your foreach loop, remember to use allRows.getAttribute("href") instead of allRows.Text in order to obtain the URL from each row.

Answer №3

namespace Test_Scraper_2
{
class ScraperProgram
{
    static void Main(string[] args)
    {
        //Set up Chrome driver
        using (var driver = new ChromeDriver())
        {
            driver.Navigate().GoToUrl("https://www.tfrrs.org/");

            //Locate elements
            var SearchField = driver.FindElementByXPath(@"/html/body/div[3]/div/div/div[4]/div/div[2]/form/div[1]/input");
            var SearchButton = driver.FindElementByXPath(@"/html/body/div[3]/div/div/div[4]/div/div[2]/form/div[4]/button");

            var Count = 1;
            //Navigate to specific page
            SearchField.SendKeys("Ashley Smith");
            SearchButton.Click();

            var rows = driver.FindElementsByClassName("allRows");   // driver.FindElementByLinkText("Ashley Smith");


            foreach (var row in rows)
            {
                var LinkNameTFRRS = row.FindElement(By.TagName("a")).GetAttribute("href");    

                Console.WriteLine(LinkNameTFRRS);
            }


            Console.ReadLine();
        }
    }
}
}

Similar questions

If you have not found the answer to your question or you are interested in this topic, then look at other similar questions below or use the search

Execute a JavaScript function that utilizes ref/out parameters to call a WebMethod

Below is a sample webmethod [WebMethod] public string CheckService(string name, ref string msg) { return "Hello" + name; } Here is the ajax call $(document).ready(function () { $.ajax({ type: "PO ...

Discovering an element within the identical class as another element using Selenium in Python

If I have already located an element, is there a method to find another in the same category? For instance: <div class="day"> <span class="day_number">4</span> <span class="day_item_time" data-day-total-time="day-total-time">1 ...

Clicking the radio button automatically triggers its click() method, causing it to toggle between being clicked and un

I encountered an issue while attempting to trigger the click event of a radio button using the click() method in Selenium WebDriver. Even though the element is successfully identified and clicked, it quickly becomes unselected due to the click being perfor ...

Tips for extracting recipients from an email

When using Outlook.Interop to extract the Html Body from received emails, I am encountering an issue where I can only retrieve the names of recipients and not their full email addresses. Application myApp = new ApplicationClass(); NameSpace ...

Tips for transforming the appearance of an asp.net page with asp:Content into a stylish css-page

Currently facing an issue where I am unable to change the background-color in my CSS file. As a workaround, I have placed the style directly within an ASP page. <asp:Content Id="Content2" ContentPlaceHolderID="ContentPlaceHolder1" Runat="Server"> ...

Unable to locate and interact with a specific element within a dropdown menu while utilizing Protractor

I am currently facing an issue with selecting a checkbox from a dropdown in jq widgets. The code seems to work fine when the element is visible on the screen, but fails otherwise. I have tried various methods such as executeScript and scrollIntoView to bri ...

The timing of the JavaScript dialog with the AJAX call is off-kilter

Encountering an issue with a JavaScript script designed to showcase a JQUERY dialog box based on a C# ViewModel. Within a repeater, there is an ASP drop-down menu displaying 'Registration Date' details. The objective is for the JavaScript dialog ...

Python, Selenium, and gecko driver with added browser extensions

Earlier today, Selenium with Firefox and extensions was working perfectly fine. However, after updating FF, Selenium stopped working (see here) which forced me to switch to geckodriver. Now, I am trying to run Selenium (gecko driver) with the udrive exten ...

Retrieving information from an Open Office spreadsheet using Selenium and Java

Can Open Office spreadsheet (ODS) files be utilized for reading and writing data in Selenium with Java? I have conducted a thorough search online but have not been able to find any solutions. Any assistance would be greatly appreciated. ...

Discovering the xpath with a certain condition based on text length using Python Selenium

In my quest to extract the specific content I need from countless pages, I have devised a reliable rule that works 99% of the time: //a[@class='popular class' and not (contains(text(),'text1')) and not (contains(text(),'text2&apos ...

Setting Timeout in Selenium IDE

Help! I've encountered an issue while using Selenium IDE. When attempting to utilize the wait for element present command, I received this error message: https://i.stack.imgur.com/WOswm.png I searched everywhere but couldn't find any options to ...

Choose a checkbox by targeting a specific column value with XPath

While automating the testing with Selenium, I encountered an issue related to selecting a checkbox in a table row. To resolve this problem, I turned to XPath for assistance. Specifically, I needed to choose the row based on the file name. Below is the rele ...

After hitting the submit button, it is important to collect the ID

When working with selenium in Java, I have encountered a scenario where clicking on the submit button generates a random ID on the screen (displayed within a div). I need to capture this generated ID and input it into another field to ensure all relevant ...

Having difficulty choosing dropdown option within iframe using Selenium Java

I am facing an issue where I cannot seem to choose a value from a dropdown menu that is located inside an iframe on the website: Below is the code that I have been using: public class SelectDropdown { public static void main(String[] args) throws Interr ...

Is there a way to navigate to the "Error" view page following an AJAX request?

Currently, I am utilizing an Ajax function to call an HTTP Post Action method located in the controller. During this process, the action method is producing a basic exception message. To address this issue, I have applied my own personalized handler attr ...

Capturing Timeout error in selenium testing

Currently, I'm utilizing the PhantomJS webdriver in a C# WinForms project. One key difference between PhantomJS and Firefox is that PhantomJS will proceed with the code execution even if the webpage has not fully loaded. To ensure it runs only when t ...

Unlock the npm package on TFS

Encountering an issue with a npm package called tfs-unlock while using grunt. Upon trying to build, the server returns the following error message: [exec] [4mRunning "tfs-unlock:checkout" (tfs-unlock) task[24m [exec] [31m>> [39mChild process exi ...

Using the powerful combination of Selenium 4.0 with Java, integrated with Cucumber, Browserstack, and Applitools for

Looking to develop a cutting-edge test automation framework with Cucumber integration, Selenium Webdriver version 4.0 offering cross-browser functionality, execution on Browserstack, and visual validation via Applitools. Open to any suggestions to make th ...

Testing the functionality of Webdriver by simulating events and notifications within the test environment

Let me paint the picture: We execute webdriver tests using Python to test a robust-client application written in Javascript. In the event of an error (typically when the backend fails to respond or responds with FAILURE), the application displays a "We&ap ...

I must interact with the video within the iframe by clicking on it

I am trying to interact with an iframe video on a webpage. Here is the code snippet for the video: <div class="videoWrapper" style="" xpath="1"> <iframe width="854" height="480" src="xxxxxxx" frameborder="0" allow="autoplay; encrypted-media" all ...