Questions tagged [r]

R is an amazing programming language and software environment that allows users to perform statistical computing, bioinformatics, visualization, and general computing. It offers complete freedom as it is a free and open-source tool. To ensure easy collaboration, please provide concise reproducible examples that others can easily run by simply copying and pasting the code. Additionally, make sure to display the desired output in its entirety. For any data-related inquiries, utilize dput() to share data, and if you are using non-base packages, remember to specify them with library(). Avoid embedding pictures for data or code; instead, use indented code blocks. If you have any statistics questions, consider visiting r for expert assistance.

What is the best way to utilize Rselenium's findElement() function in order to target an xpath based on its text content

Despite researching similar questions on stack overflow, I have not been able to make my code work. Here is the initial code I am working with: library(RSelenium) library(rvest) remote_driver <- RSelenium::remoteDriver( remoteServerAddr = "local ...

Eliminating the occurrence of the dot character within column names in R while employing a JSON dataset

I have been attempting to tidy up the column names in my R code. The data I am working with is from a JSON dataset that I imported into R using the "stream_in" function from the jsonlite package. Initially, I tried using both the "gsub" command and the "p ...

Python ModuleNotFound error when invoking Python code from system() function in R

I have a scenario where I have an R script running on my Raspberry Pi that calls a Python script using the system() function. Let's call this R script tweetsomething.R # first some data cleaning etc. system("python3 tweetImage.py var1 var2" ...

Creating One Comprehensive Object by Merging Multiple Character Objects

I am attempting to extract text from a news article using the code snippet below: library(rvest) url <- "https://www.bbc.com/future/article/20220823-how-auckland-worlds-most-spongy-city-tackles-floods" final = url %>% read_html() %>% ...

There was a problem encountered while trying to import Json data

I'm having trouble importing my JSON file into R. I've installed the necessary packages and here's what I attempted: library(rjson) JsonData <- fromJson(file= "<filename.json>") I also tried using the file path and the jsonlite pa ...

Saving JSON data within a List

Currently, I am working on scraping a series of websites with similar formats such as www.website1.com, www.website2.com, www.website3.com, and so on. In order to extract JSON data from each website, I have implemented a code snippet that includes a "time ...

Minimize the expansive width of Shiny Dashboard's sidebar for

I'm facing a challenge in hiding a shinydashboard sidebar with a wider width. Upon increasing the width, however, the sidebar isn't completely disappearing from the screen. Here's an example that illustrates the issue I'm trying to address; ## R Code ## l ...

Utilizing Rvest for extracting data from LinkedIn profiles

I've been trying to scrape my LinkedIn profile using Rvest, but I encountered a problem in the experience section. The XPath below was supposed to retrieve the experience section, but it's returning 0 nodes: Test <- read %>% html_nodes(xpath = & ...

Reading a complex json file with multiple levels in R

While I have a strong grasp of R, I am relatively new to JSON file formats and the best practices for parsing them. My current challenge lies in constructing a data frame from a raw JSON file that contains repeated measure data with multiple observations p ...

Resolving formatting challenges in R markdown with Blastula

Dealing with formatting problems while trying to include an R markdown file in an email body. As a CSS novice, I'm struggling to find a solution and it's becoming quite frustrating. The table has the following CSS styling: <!DOCTYPE html ...

Is it common for NA values to appear in the data frame due to JSON parsing?

I recently obtained a collection of JSON files from the YELP public data challenge. These files can be found at this link: The files are in NDJSON format, and I have successfully read them using the following code: library(jsonlite) df <- stream_in(fi ...

Change the color of the text in a shiny dashboard

While exploring shiny dashboard, I came across this link that explains how to modify colors in the sidebar and header. However, I'm struggling to change the font color for sidebar items. The default white font color becomes unreadable when using light colo ...

Unable to launch web browser with RSelenium

I'm looking to utilize Selenium for web scraping in R. Operating System: Windows 11, version 21H2 I have updated Java to the latest version (1.8.0_351). Mentioning it as it may be a solution in these cases. However, I encounter an error when definin ...

Convert the dataframe into a JSON format in the form of a

Looking for assistance in converting the given dataframe to JSON format using R. I have tried multiple methods but haven't been successful. Any help would be greatly appreciated. Sample code: df <- data.frame(month = c(1, 1, 1, 2, 2, 2), ...

Unexpected error occurs when executing a valid MongoDB query, displaying 'Error: Invalid JSON object'

Problem I seem to have made a mistake in the JSON query syntax, but I can't figure out where. My goal is to group the data in overviewData based on four values and add a COUNT feature. Solution overviewData <- M_CONNECTION$aggregate('[ { ...

A guide on adding a JSON object to an array in R

I have attempted various methods to insert a JSON object into an array and save it in the same format as the example below, but unfortunately, I have not been successful. Does anyone have a solution to accomplish this in R? Thank you EDIT : I have foun ...

Is there a way to convert dates and times in R similar to strtotime

Have you ever heard of PHP's amazing function, strtotime? It can take any string with almost any date format and convert it into a time value (count of seconds since 1970). What I love about it is its future-proof nature compared to functions like str ...

Is it possible to execute custom JavaScript code in an R Jupyter notebook?

Within my Jupyter Notebook, I am working with the R programming language and would like to integrate javascript functions into it. I'm aware that there are libraries in javascript that can be called from R, but I haven't been able to find any examples of ...

Can you identify the HTML table format and suggest effective web scraping methods?

I have been attempting to retrieve data from the following link, http://www.rchsd.org/doctors/index.htm? strt = 0 & ln = & fn = & sp = & grp = & loc = & lng = & gen = , using R but finding it quite challenging. I have observed that the URL remains constan ...

Extracting all Twitter links from a webpage with RSelenium

I'm encountering an issue while attempting to extract URLs from a webpage using Rselenium. I keep receiving an InvalidSelector error. My setup consists of R 3.6.0 on a Windows 10 PC, with Rselenium 1.7.5 and Chrome webdriver (chromever="75.0.3770.8") ...

Parsing clean data from intricate JSON structures in the R programming language

Utilizing the tidyjson package makes extracting neat data from a simple JSON effortless () However, when it comes to dealing with a more intricate nested JSON structure, applying this method becomes challenging. Specific questions like this one (how do yo ...

Using R to retrieve values from JSON lists

My knowledge in using R is limited and I need to create a script for a school project. I have a json file with nested lists, and my task is to extract values from two specific attributes. The challenge lies in the fact that these attributes are located i ...

two examples of time series datasets in R and Python

Due to my inclination towards efficiency, I prefer not to waste time on downloading datasets, loading them, and preprocessing just to test some basic functions on various timeseries data. Can you suggest any readily available sample timeseries datasets i ...

Generate dynamic maps that are exportable to HTML format

I've utilized Leaflet to produce engaging interactive maps in R. However, I'm encountering an issue when trying to export the maps - the background map appears grey after exporting. library(leaflet) library(htmlwidgets) m <- leaflet(data.frame(lat = 5 ...

Attempting to identify and automatically fill out a form using RSelenium

I am currently facing an issue with detecting and populating elements in an embedded form on my website using RSelenium. The form code is: <div id="form"> <form accept-charset="utf-8" method="POST" novalidate=""> ...

Optimal method for shuffling a matrix in either R or Python

Currently, I am dealing with a large numeric matrix M in R that consists of 11000 rows and 20 columns. Within this matrix, I am conducting numerous correlation tests. Specifically, I am using the function cor.test(M[i,], M[j,], method='spearman' ...

How should I integrate my JS authentication function in Rshiny to enable the app to utilize the outcome?

Currently, I have an Rshiny application set to be published on the server but in order to ensure that a specific user has access, we require an API authentication token. The process of authentication is handled within JS tags outside of the application, wh ...

How to Enhance HTML Requests with a Variety of Search Terms

I recently came across a helpful post on how to use R to search for news articles on Google. The post provides a link to Scraping Google News with Rvest for Keywords. The example in the post demonstrates searching for a single term, such as: keyword <- ...

What is the best way to handle a nested list() that arises while parsing JSON files in R?

Having multiple JSON files that need to be read and combined in r, each containing data for 51 observations. However, upon reading the JSON file in r, the required information is nested within the "mentions" column. The goal is to extract the time stamp fr ...

Utilizing personalized CSS for specific ioslides

If I decide to stick with the default data and presentation of a new ioslides, changing only the second slide (## R Markdown) without affecting the entire item, how can I do this by setting up the css document accordingly? My goal is to simply adjust the f ...

Performing G-testing in R and Python (Comparing Proportions between Two Samples)

While conducting the G test in both R and Python, I have come across different results. Surprisingly, the results from my Python code seem to be inaccurate, suggesting there might be a misapplication of the formula. The dataset is as follows: prfs Sex F ...

Changing JSON keys to lowercase and eliminating special characters in RConverting JSON key names to lowercase

Exploring the jsonlite package in R has brought me to a point where I need to convert a complex list into a JSON object. Let's say we have a list named x: library(jsonlite) x = list(a = 1, B = 2, c = list(D = 4, e = 5, 'F g' = 'NAME', H = list(i = list(j ...

Exploring the Discrepancies in Augmented Dickey-Fuller Test Outputs between R and Python

What could explain the significant difference in results between the ADF test in R and Python? Dataset: The R test indicates that the time series is stationary, while the Python test suggests it is non-stationary. Python df = pd.read_csv("../data/ai ...

Achieving Text Centering in kableExtra Cells with CSS: A Step-by-Step Guide

Recently, I received a helpful solution that involved using the extra_css = argument in cell_spec() from the kableExtra package. This allowed me to fill the entire cell of my table instead of just the text inside it. However, even after specifying align = ...

RSelenium: What is preventing me from inputting a password into this (Java?) login form?

In the midst of my web scraping endeavors, I have encountered a slight hiccup when trying to extract data from this particular website using RSelenium. While I've had success sending text to various elements on the page, the login field has proven to ...

eliminate whitespace characters within a string

Here is a question that suggests an easy way to remove space characters in a string using R. However, I encountered an issue when trying to remove a space between two numbers (e.g. 11 846.4) while working with the following table: require(XML) require(RCur ...

Combining various columns using different functions in Pandas

Python's Pandas and R's Dplyr are both powerful tools for handling data. In R, using dplyr allows you to achieve the following in a concise manner; custom_func <- function(col1, col2) length(col1) + length(col2) ChickWeight %>% group_by ...

Tips for creating an HTML report using Python

Currently, I am searching for a solution to convert my saved matplotlib graphs as png files along with some data frames into HTML format. This is similar to how I typically use R2HTML in R. Despite my efforts, I have not been able to locate clear informat ...

Spotfire: Changing a URL in an input field after it has been entered by the user

Currently, I am faced with a challenge related to importing a file in spotfire based on the path provided by the user. To accomplish this, I am utilizing a data function written in R script. In R, it is essential to note that all "" characters are not re ...

Tips for inputting text directly into website search bars using the R language

Can anyone assist me in finding a method to perform web scraping on a webpage post typing something into its search box? To illustrate, I am seeking an R function capable of typing the term "notebook" directly onto Amazon's homepage, enabling me to condu ...

Utilizing CSS styling to create page breaks in R Markdown disrupts the flow of table of contents

Encountering difficulties in achieving a visually pleasing document using knitr's export to pdf, I have resorted to converting the html file for my Markdown project to pdf through the wkhtmltopdf command line utility tool. A simplified Markdown docum ...

Panel div fails to show visNetwork visualization

After experimenting with shinyLP for creating html elements and crafting network diagrams using visNetwork, an interesting observation was made. When visNetwork is placed in a well panel or without any panel at all, it displays properly. However, the issue ...

python pandas: the alternative to R's dcast command

I want to achieve the equivalent of the commands below in Python: test <- data.frame(convert_me=c('Transform1','Transform2','Transform3'), values=rnorm(3,45, 12), age_col=c('23','33','44')) test library(reshape2) t <- dcast(test, ...

Exploring the world of web scraping using R's XML package with the powerful xpathS

I need help extracting the names of shopping malls from a specific website. The URL is provided here: After examining the html code, I noticed that the mall names are stored under the "h5" tag. I tried to extract the text using the following codes, but it ...

Encountering Problem Importing HTML Table Data into R

I am looking to import the data table located at the bottom of a specific webpage into R, either as a dataframe or table. The webpage in question is . Initially, I attempted to utilize the readHTMLTable function from the XML package: library(XML) url <- ...

Creating a buffer for data-axis in matplotlib using the R style approach

In R plots, the x and y limits are automatically adjusted to add some space between the data and the axes. I am curious if matplotlib has a similar feature to do this automatically. If not, is there a specific formula or rule of thumb that R uses to dete ...

R: Managing JSON-formatted error responses

My task involves retrieving user details from the Facebook Graph API. Here is an example of how I am doing it: require(RJSONIO) response <- RJSONIO::fromJSON("http://graph.facebook.com/?ids=Jack") print(response) # $Jack # id first_name ...

Exploring the limitations of assigning a variable value in the error function of a tryCatch (R)

As I work on developing a web scraping script, I often encounter errors that require me to handle exceptions. Despite using a tryCatch function to catch these errors, I still see error messages popping up. To suppress these messages, I have used the suppre ...

Tips for eliminating fade effect during hover in networkD3 chart in R

Currently, I have been exploring the usage examples of networkd3 in r I am curious if there is a way to eliminate the hover effect where everything else fades when hovering over a specific node in the graph. For reference, check out "Interacting with igra ...

Adding a dynamic object to the secondary column header of an output table: A step-by-step guide

Working on a transition table module, my main challenge is presenting the output in a user-friendly manner. In the past, I used Excel for this task, where making the table readable was simple but extracting data from millions of rows took hours. Now with R ...

How to access a shadowroot element using R Selenium

Currently, I'm dealing with web scraping where one of the websites presents a 'Accept cookies' button within a shadow root. To address this issue, I sought assistance on how to click this button in discussions found here: Click on accept cookie buton with ...

Finding the root cause of the error message 'Unexpected character: :' from using rjson

My company recently acquired a JSON data set, and I need to extract specific information from it. However, when attempting to import the data using the "fromJSON" method, I encountered an error as described in the title. With over 16,000 files worth of d ...

Using the FileInput Shiny App to Upload JSON Files

I'm currently working on developing a shiny app that can load and display data from a .json file in a table format. I've attempted to import the data and present it using DataTable along with the fromJson function from Json Lite. options(shiny ...

Challenge implementing custom javascript to display categorical/string features on Shiny slider

I'm attempting to design a unique Shiny slider that represents the months of the year. My desired outcome is for the slider to display the names of the months as strings, rather than numeric values where 1 corresponds to January, 2 corresponds to February, ...

Generating a dataframe with cells that have a combination of regular and italicized text using the sjPlot package functions

I've been trying to set up a basic data table where the genus name in the "Coral_taxon" column is italicized, but the "spp." part following it remains lowercase. I thought of using the expression() function for each row in "Coral_taxon," but so far, I have ...

Glistening tabPanel and Analytics by Google

I recently completed a comprehensive tutorial here that delves into the intricacies of Google Analytics. Despite grasping the concepts explained in the tutorial, my understanding of jQuery remains at ground zero. In my ShinyApp, I utilize multiple tabPanel ...

Change a JSON file of Firefox bookmarks into markdown format

Context I'm looking to display a selection of my bookmarks on my Hugo website, sourced from Firefox in JSON format. I want the output to reflect the nested structure through a nested list, tree view, or accordion format. The content on my website is ...

Using R for web scraping, I encountered a unique situation where the 'next page' button was only programmed to navigate to the top of the page

I am attempting to extract the headline snippets from a local newspaper's articles by utilizing rvest and RSelenium. To access pages beyond the initial one, I need to click on the 'next page' button. Strangely, when I perform this action through RSelenium, ...

Change the name of a series of JSON objects

Among my JSON entities, there is one called 'info'. Take a look at it: null null { "id": "qwefhu214o", "number": "2346", "date": "28.01.2019" } null null { "id": " ...

Converting JSON data into a dataframe yields varying numerical values

Consider the following JSON data: $fraudster [1] FALSE $actionType [1] "LOGIN_FORM" $actionId [1] 10 $eventList [1] "[[1544018118438041139,162.0,38.0,0.023529414,1.0,2131230815,1], [1544018118466235879,162.0,38.0,0.025490198,1.0,2131230815,0], [1544018 ...

What is the process for taking individual files from a folder one by one, processing them, and then moving them to another folder?

I have a script written in Python called `cleaning_script.py` which is used to pre-process individual CSV files. Currently, I am running the script by executing this command in my Rstudio terminal: python3 cleaning_script.py file1.csv results_file1.csv. Th ...

Tips for transforming gridded temperature data in CSV format (organized by latitude and longitude) into a raster map

Recently, I acquired a dataset that shows the average temperature change across the entire US, but it is formatted with lat/long range. The original csv file can be found here: original csv I am attempting to convert this data into a raster for visualizat ...

RSelenium in conjunction with Chrome fails to render the entire webpage

Struggling to navigate using RSelenium, I am faced with missing element text and incomplete page loading. This issue seems to be specific to Chrome, as it works fine on FireFox. An example code snippet: library(RSelenium) rD <- rsDriver(port = 4567L, ...

Transitioning from Python to HTTR

When it comes to authenticating myself on the api using python, I have a set of parameters that I can use: library(reticulate) import requests url = 'https://api.checkbox.com/v1/my_account/oauth2/token' payload = 'username=my_username& ...

Adjust a Javascript script to choose the best font color for use in R Shiny applications

I am currently seeking to determine the font color of hover messages based on the background color. This means white if the background is dark, and black if it is light. However, I stumbled upon a Stack Overflow question with a Javascript solution that see ...

I am looking to transform intricate json data into csv format with the help of either python or R programming

Converting values to rows in CSV and keys to columns in CSV format would be beneficial. { "_id": { "$uId”: “12345678” }, “comopany_productId”: “J00354”, “`company_product name`”: “BIKE 12345”, "search_resu ...

Troubleshooting Rpy2: Missing R.dll File

Operating on Windows 7 64bit with Python version 2.7.9, I have successfully installed R-3.2.0. Despite trying previous versions such as R-2.12.2 and R-3.1.3, the result remains the same. To ensure compatibility, I added C:Program FilesRR-3.2.0ini386 ...

The process of exporting a dataframe to JSON in a particular format

Is there a way to modify the format of a json file exported using a downloadButton in a Shiny App? The current code successfully exports a data frame to json, but I want it to have a different structure. Here's the code that currently works: library(s ...

Need help fixing a corrupted JSON file that contains escaped single and double quotes?

Challenge I am faced with a sizable JSON file (~700,000 lines, 1.2GB in size) that contains Twitter data which needs preprocessing for both data and network analysis purposes. While collecting the data, an error occurred: instead of using " as a separator ...

Using R programming language to download PDF files from a website through web scraping

Looking for assistance in R coding to automatically download all the PDFs linked on this URL: . The goal is to save these PDFs into a designated folder. I've attempted the following script with guidance from , however, it's resulting in errors: library(tid ...

Is it possible for a Python and R file to collaborate and share a single webdriver session?

I've been working on a web scraper using RSelenium, but have found that certain tasks are better suited for Python. To streamline this process, I've created a .Rmd file with code snippets in both R and Python. The RSelenium side of the project i ...

Troubleshooting difficulties integrating NodeJS with R using an R script

I have been attempting to execute an R-script from a js-file but I am consistently receiving a null value as the return. Here is my folder structure: -R_test --example.js --helloWorld.R The contents of example.js are as follows: var R = require(" ...

Conceal or style the filter panel in a DT datatable

Here is an example to consider: library(DT) L <- 10 datatable( data.frame( var1 = sapply(1:L, function(x) paste("<X>",paste0(x, letters, LETTERS, "\n", ...

Analyzing intricate JSON data as a data frame in R

Here is the JSON data that I have: json_data <- data.frame(changedContent=c('{"documents":[],"images":[],"profileCommunications":[],"shortListedProfiles":[],"matrimonyUser":{"createdBy":null,"parentMatrimonyUserId":0,"userSalutationVal":"Mr.","mat ...

Understanding JSON in R by extracting keys and values

I have a JSON URL that I need to parse in R. The URL is https://{{API_HOST}}/api/dd/new and contains both keys and values. I can easily parse this JSON in Postman by using the keys and values in the Headers section. Now, I am looking for a way to achieve t ...