Page tree
Skip to end of metadata
Go to start of metadata

 

Supported Data Types

Format / System

Importing and Exporting

Supported Compression

Product tested version

Datameer version availability

Notes
ImportExport.gz.bz2.lzoSnappy.zip.ZImportExport
Amazon Redshift(tick) (tick)       8.2, 8.45.55.5Native Amazon Redshift JDBC 4.1 driver or a PostgreSQL jdbc driver can be used.
Apache Avro(tick)(tick)  (tick)(tick)(tick)  2.1.42.1.4 
Apache Web Server Logs(tick)(error)          
Azure Blob Storage(tick)(tick)       4.04.0

(For HDP2.0, 2.2 and CDH+4 users)

Contact Datameer services for info.

Cassandra(tick)(error)      0.6.5  Public available plugin, see https://github.com/zznate/datameer-cassandra-plugin
Cobol Copybooks(tick)(error)          
CSV/TSV, etc.(tick)(tick)  (tick)(tick)(tick)     
DB2(tick)(tick)      

8.1 (since 1.3.9), 9.7 Express-C

   
Excel Workbooks(tick) (tick)      2007 and after2.1.42.1.4 
Facebook Graph API (files)(tick)(error)          
Fixed Width Text(tick)(error)          
Google Spreadsheets(tick)

(error)

       2.1  
Greenplum(tick)(tick)       HD 1.1, HD 2.1, 4.1.1.1   
HBase(tick)(error)      0.98.x, 0.96.x, 0.94.x, 0.92.x, 0.90.x  

In order to satisfy the classloader requirements, hbase-protocol.jar must be included in Hadoop's classpath and the root Datameer classpath (/etc/custom-jars) for version 0.96.1 to 0.98.0

Hive Metastore(tick) (tick)       0.13, 0.12, 0.11, 0.10, 0.9, 0.8, 0.7  Export file format: org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat
HiveServer2(tick) (tick)       

0.13, 0.14, 1.1.0.x

5.9  
HSQL-DB(tick)(tick)          
HTML(tick) (error)       3.1  
JSON(tick) (error)          
Log4j Log File(tick) (tick)        2.1.52.1.5 
MBOX (email archive files)(tick)(error)          
MS IIS Web Server Logs(tick)(error)          
MSSQL(tick)(tick)      SQL Express 2005 and 2008   
MySQL(tick)(tick)      5.5, 5.6, 5.7   
Netezza(tick) (tick)       6.0.6   
Oracle(tick)(tick)      10g XE, 11g XE   
OpenStack Swift(tick)(tick)       5.115.11 
ORC (Optimized Row Columnar)(tick)(error)  (tick)(tick)(tick)   4.5, 5.1 

Only supported in conjunction with Hive

Parquet(tick) (tick)   (tick)(tick)(tick) Parquet 2.15.35.3 
PostgreSQL(tick) (tick)       9.0.x   
RCFile (Record Columnar File)(tick) (error)          
Sequence Files with Metadata(tick)(error)  (tick)(tick)(tick)     
Spark(tick)(tick)       5.105.10 
Sybase IQ(tick) (tick)       12.7   
Tableau(tick)(tick)        5.7 
Teradata(tick) (tick)      12 & 13  Teradata database needs to be configured to support the appropriate character set.
Teradata Aster(tick)(tick)      5.0   
Text Files(tick)(error)  (tick)(tick)(tick)     
Twitter Firehose (files)(tick)(error)          

Vertica

(tick)

(tick)

      

5, 6.0, 6.1

 

  

XML

(tick) 

(error)

  (tick)(tick)(tick) 

 

 

  

 

Supported File Protocols

Protocol

Input

Output

File

(tick)

(tick)

HDFS

(tick)

(tick)

SSH (SCP and SFTP)

(tick)

(tick)

S3

(tick)

(tick)

S3_BLOCK

(tick)

(tick)

As of v3.1, Datameer supports Bitverse SSH Server/Client for the Windows platform. The root paths to be specified while creating the connection should look something like: /c:/mydata/folder1

Supported File Compression Codecs

Codec

Input

Output

.gz

(tick)

(error)

.bz2

(tick)

(error)

.lzo

(tick) (* additional native libraries required)

(tick)

Snappy(tick)

(error)

.zip

(tick)

 (tick)
.Z(tick)(error)

* LZO: tested with Fedora lzo.i386 v2.02-3.fc8 and http://github.com/kevinweil/hadoop-lzo (state from 2010-Jun-20)

Supported File Systems

Customer

File system

Special Hadoop configuration

appistry

storage:/

fs.storage.impl=org.apache.hadoop.fs.appistry.FabricStorageFileSystem
fs.abs.impl=org.apache.hadoop.fs.appistry.BlockedFabricStorageFileSystem
fs.appistry.storage.host=localhost
fs.appistry.chunked=false
fs.appistry.jetty.port=8085

  • None