Page tree
Skip to end of metadata
Go to start of metadata

Input means you can import data of that type. Output means you can export data to that type.

Supported Data Types

Format / System

Input

Output

Tested version

Production version

Notes

Text Files

(tick)

(error)

 

 

 

CSV/TSV, etc.

(tick)

(tick)

 

 

 

Apache Web Server Logs

(tick)

(error)

 

 

 
MS IIS Web Server Logs(tick)(error)   
Twitter Firehose (files)(tick)(error)   
Facebook Graph API (files)(tick)(error)   
Fixed Width Text(tick)(error)   

JSON

(tick) (since 1.2.6)

(error)

 

 

 

XML

(tick) (since 1.2.6)

(error)

 

 

 

MBOX (email archive files)

(tick)

(error)

 

 

 

Hive

(tick)

(tick)   (since 2.0.1)

0.12, 0.11, 0.10, 0.9, 0.8, 0.7

0.12, 0.11, 0.10, 0.9 , 0.8, 0.7

 

HBase

(tick)

(error)

0.94.x, 0.92.x, 0.90.x

0.94.x, 0.92.x, 0.90.x

 

Cassandra **

(tick)

(error)

0.6.5

0.6.5

 

MySQL

(tick)

(tick)

5.1, 5.5*, 5.6

5.1, 5.5*, 5.6

MySQL 5.6 is available in Datameer v2 from 2.1.7.14 and up.

Oracle

(tick)

(tick)  

10g XE, 11g XE

10g, 11g

 

DB2

(tick)

(tick)

  • 8.1 (since 1.3.9)
  • 9.7 Express-C

 

 

MSSQL

(tick)

(tick)

SQL Express 2005 and 2008

SQL 2000, 2005, 2008

 
PostgreSQL(tick) (since 1.4)(tick) (since 1.4)9.0.x  

HSQL-DB

(tick)

(tick)

 

 

 

Teradata

(tick) (since 1.1.3)

(tick) ***

12 & 13

 

 
Teradata Aster(tick) (since 2.1.6)(tick) (since 2.1.6)5.05.0 
Netezza(tick) (since 2.0.1)(tick) (since 2.0.1)6.0.6  
Vertica(tick) (since 1.4.4)(tick) (since 1.4.4)5  
Greenplum(tick) (since 1.4)(tick) (since 1.4)HD 1.1, HD 2.1, 4.1.1.1HD 1.1, HD 2.1, 4.1.1.1 

Sybase IQ

(tick)   (since 2.0.1)

(tick)   (since 2.0.1)

12.7

 

 
Excel Workbooks(tick)   (since 2.1.4)(tick)   (since 2.1.4)2007 and after2007 and after 
Log4j Log File(tick)   (since 2.1.5)(tick)   (since 2.1.5)   

* Make sure you are using Connector/J v5.1.26 or newer because of http://bugs.mysql.com/bug.php?id=38747
** public available plugin, see https://github.com/zznate/datameer-cassandra-plugin
*** Teradata database needs to be configured to support the appropriate character set.

Supported File Protocols

Protocol

Input

Output

File

(tick)

(tick)

HDFS

(tick)

(tick)

SSH (SCP and SFTP)

(tick)

(tick)

S3

(tick)

(tick)

S3_BLOCK

(tick)

(tick)

Versions of Datameer 2.1.7 and before do not support SSH/SCP for Windows. SFTP is supported for Windows. 

As of v2.1.8, Datameer supports Bitverse SSH Server/Client  for the Windows platform. The root paths to be specified while creating the connection should look something like: /c:/mydata/folder1

Supported File Compression Codecs

Codec

Input

Output

.gz

(tick)

(error)

.bz2

(tick)

(error)

.lzo

(tick) (* additional native libraries required)

(error)

Snappy(tick) (only with Clouderas CDH)(error)

.zip

(tick)

(error)

.Z(tick)(error)

* LZO: tested with Fedora lzo.i386 v2.02-3.fc8 and http://github.com/kevinweil/hadoop-lzo (state from 2010-Jun-20)

Supported Filesystems

Customer

File system

Special Hadoop configuration

appistry

storage:/

fs.storage.impl=org.apache.hadoop.fs.appistry.FabricStorageFileSystem
fs.abs.impl=org.apache.hadoop.fs.appistry.BlockedFabricStorageFileSystem
fs.appistry.storage.host=localhost
fs.appistry.chunked=false
fs.appistry.jetty.port=8085

  • No labels