hdfs

Another feature that I would like to have in smart_open is support for direct reading/writing of the data to/from ftp.

Anyone else interested in this? Does this make sense?

Currently, when calling pytest, we use -m "not udf" to skip backends with missing or broken udf support. For all other features, we've been using a decorator @pytest.mark.skip_backends([SQLite], reason='not supported')` instead.

It would be good to skip udf tests for backends that don't implement them with the decorator. So, calling the tests for a backend skips them automatically if the back

Similar to how unix ls works, param could be -t

Given the new key-value store event stream, it'd be nice to have something like:

$ skein kv events <application id> [options...]

where the process blocks, and logs the event stream to the console until interrupted. This would be useful for debugging, as well as demos.

Assuming a valid hive-site.xml, it will be possible to determine the active hive warehouse HDFS directory and HiveServer2 and Metastore URIs.

From there we should be able to perform a directory analysis on the hive warehouse parent directory and then all HDFS locations that represent tables / partitions.

Aug	SEP	Oct
	05
2019	2020	2021

hdfs

Here are 564 public repositories matching this topic...

chrislusf / seaweedfs

heibaiying / BigData-Notes

wangzhiwubigdata / God-Of-BigData

RaRe-Technologies / smart_open

ibis-project / ibis

TileDB-Inc / TileDB

colinmarc / hdfs

spotify / snakebite

CheckChe0803 / BigData-Interview

Stratio / sparta

sunnyandgood / BigData

confluentinc / kafka-connect-hdfs

lensesio / kafka-connect-ui

HariSekhon / DevOps-Python-tools

divolte / divolte-collector

uber / storagetapper

mtth / hdfs

mesosphere / dcos-commons

mullerhai / HsunTzu

wradlib / wradlib

tirthajyoti / Spark-with-Python

linkedin / dynamometer

fabiogjardim / bigdata_docker

avast / hdfs-shell

jcrist / skein

gglinux / wifi

PaddlePaddle / ElasticCTR

paypal / NNAnalytics

mmolimar / kafka-connect-fs

confluentinc / camus

Improve this page

Add this topic to your repo