Feather Linux Error Loading X
Contents |
Sign in Pricing Blog Support Search GitHub This repository Watch 83 Star 1,036 Fork 77 wesm/feather Code Issues 60 Pull requests 9 Projects 0 Pulse Graphs New issue IO
Feather Python
error reading multiple Feather files in R #177 Closed lmullen opened this Issue feather r Jun 3, 2016 · 6 comments Projects None yet Labels None yet Milestone No milestone Assignees feather format No one assigned 3 participants lmullen commented Jun 3, 2016 I have 18,500 Feather files (all with the same columns and column types) which I want to read in. So I do
Feather Github
this: library(feather) library(purrr) library(dplyr) paths <- Sys.glob("/media/lmullen/data/chronicling-america/out/*.feather") read_df <- failwith(NA, function(x) { message(x) read_feather(x) }) raw_l <- paths %>% map(read_df) names(raw_l) <- paths When I run that code, some number of the Feather files (it varies from about 150 to 1500) fail to load with this error. /media/lmullen/data/chronicling-america/out/sn85042907-1919.feather Error : IO error: Unable to open file There is nothing actually wrong with those Feather files
Cran Feather
though. I can read_feather(path_to_problem_file) and get back a data frame as expected. I can also get the paths which failed to load, then run map(paths_that_didnt_load, read_feather) and load all of them fine. My only suspicion is that Feather is too fast---that it reads the files so quickly that the disk can't get to the next file in time. FWIW, the files that don't load in the batch tend to come in sequence. The files are stored on a RAID 10 array, so it's not as fast as an SSD, but it's fast. When I put a Sys.sleep(1) call in between loading each file, that cuts down on the number of errors. I can't think of a good way to provide a reproducible example, but happy to do so if you can give instructions. > sessionInfo() R version 3.3.0 (2016-05-03) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04 LTS locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8 [5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8 LC_PAPER=en_US.UTF-8 LC_NAME=C [9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] readr_0.2.2 purrr_0.2.1 feather_0.0.1 dplyr_0.4.3 loaded via a namespace (and not attached): [1] pryr_0.1.2 lazyeval_0.1.10 magrittr_
Get Kubuntu Get Xubuntu Get Lubuntu Get UbuntuStudio Get Mythbuntu Get Edubuntu Get Ubuntu-GNOME Get UbuntuKylin Ubuntu Code of Conduct Ubuntu Wiki Community apache feather Wiki Other Support Launchpad Answers Ubuntu IRC Support AskUbuntu Official Documentation User
Error Loading Operating System Usb Boot
Documentation Social Media Facebook Twitter Useful Links Distrowatch Bugs: Ubuntu PPAs: Ubuntu Web Upd8: Ubuntu OMG! Ubuntu Ubuntu fether Insights Planet Ubuntu Activity Page Please read before SSO login Advanced Search Forum The Ubuntu Forum Community Ubuntu Official Flavours Support Installation & Upgrades [SOLVED] "Error loading operating system" Having https://github.com/wesm/feather/issues/177 an Issue With Posting ? Do you want to help us debug the posting issues ? < is the place to report it, thanks ! Page 1 of 2 12 Last Jump to page: Results 1 to 10 of 17 Thread: "Error loading operating system" Thread Tools Show Printable Version Subscribe to this Thread… Display Linear Mode Switch to Hybrid https://ubuntuforums.org/showthread.php?t=1397248 Mode Switch to Threaded Mode February 3rd, 2010 #1 Arachan View Profile View Forum Posts Private Message Just Give Me the Beans! Join Date Feb 2010 Location New Zealand Beans 76 DistroXubuntu 11.10 Oneiric Ocelot "Error loading operating system" Hello, I'm new to both Ubuntu and this forum, so please try to bear with me. I have a 500GB HDD with these partitions Screenshot.png I'm currently using the live CD to boot because when I try to boot to this HDD it simply comes up with the error "Error loading operating system". It was working fine until I started reformatting a different drive with XP on it; since then it has had the error. As far as I know I need to reconfigure GRUB to boot to the correct partition, but I do not know how to do this. I know that there are already threads about booting/GRUB problems. I have looked at a lot of them but because no one has exactly the same problem as me I do not know what to do. I would really appre
past January, we (Hadley and Wes) met and discussed some of the systems challenges facing the Python and R open source communities. In particular, we wanted to see if there were some opportunities to collaborate on tools https://blog.rstudio.org/2016/03/29/feather/ for improving interoperability between Python, R, and external compute and storage systems. One thing that struck us was that while R’s data frames and Python’s pandas data frames utilize very different internal memory representations, they share a very similar semantic model. In both R and Panda’s, data frames are lists of named, equal-length columns, which can be numeric, boolean, and date-and-time, categorical (factors), or string. Every column can have missing values. Around this error loading time, the open source community had just started the new Apache Arrow project, designed to improve data interoperability for systems dealing with columnar tabular data. In discussing Apache Arrow in the context of Python and R, we wanted to see if we could use the insights from feather to design a very fast file format for storing data frames that could be used by both languages. Thus, the Feather format was born. What is feather linux error Feather? Feather is a fast, lightweight, and easy-to-use binary file format for storing data frames. It has a few specific design goals: Lightweight, minimal API: make pushing data frames in and out of memory as simple as possible Language agnostic: Feather files are the same whether written by Python or R code. Other languages can read and write Feather files, too. High read and write performance. When possible, Feather operations should be bound by local disk performance. Code examples The Feather API is designed to make reading and writing data frames as easy as possible. In R, the code might look like: library(feather) path <- "my_data.feather" write_feather(df, path) df <- read_feather(path) Analogously, in Python, we have: import feather path = 'my_data.feather' feather.write_dataframe(df, path) df = feather.read_dataframe(path) How fast is Feather? Feather is extremely fast. Since Feather does not currently use any compression internally, it works best when used with solid-state drives as come with most of today’s laptop computers. For this first release, we prioritized a simple implementation and are thus writing unmodified Arrow memory to disk. To give you an idea, here is a Python benchmark writing an approximately 800MB pandas DataFrame to disk: import feather import pandas as pd import numpy as np arr = np.random.randn(10000000) # 10% nulls arr[::10] = np.nan df = pd.DataFrame({'column_{