How to create hive comparable table statement using Python

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • manyatha
    New Member
    • May 2017
    • 3

    How to create hive comparable table statement using Python

    I/p:SQL server file is input
    I/p:Hive create statements
    I tried with this code but unable to reach some portion.please help and am new to Python
    Eg create statement

    create external table db.emp
    (
    Name string,
    I'd int,
    Fname string,
    .....,

    )
    ROW FORMAT DELIMITED FIELDS TERMINATED BY '~'


    LOCATION 'usr/bin/...';


    #!/bin/sh
    import string
    from itertools import izip

    infile = r"ip1.txt"
    outfile = "op1.txt"
    delete_list = ["NULL,","NULL", chr(91),chr(93) ,chr(41),chr(40 ),chr(44)]
    datatypes = ["varchar","date time","char","n umeric"]
    fin = open(infile)
    fout = open(outfile, "w+")
    for line in fin:
    for word in delete_list:
    line = line.replace(wo rd, "")
    out0= line.strip(' \t\n\r')
    out1= out0.split(' ')
    for word in datatypes:
    if 'varchar 'in out1[1]:
    line = line.replace(wo rd, "string")
    elif 'numeric' in out1[1]:
    line = line.replace(wo rd, "int")
    elif 'char' in out1[1]:
    line = line.replace(wo rd, "string")
    elif 'datetime' in out1[1]:
    line = line.replace(wo rd, "timestamp" )
    else:
    line = line.replace(wo rd,"string")
    fout.write(line )
    fin.close()
    fout.close()
    f1 = open("op1.txt", "r")
    temp = open("ip2.txt", "w")
    temp1 = open("ip3.txt", "w")
    f2 = open("op2.txt", "w")
    for line in f1:
    out0 = line.strip(' \t\n\r')
    out1 = out0.split(' ')
    x = out1[1]
    print >>temp,x
    y = out1[0]
    print >>temp1,y
    f1.close()
    temp.close()
    temp1.close()
    with open("ip2.txt") as f3:
    z = ''.join(i for i in f3.read() if not i.isdigit())
    with open("op2.txt", "w")as f2:
    f2.write(z)
    with open ('op3.txt','w') as res:

    with open('ip3.txt') as f1:

    with open('op2.txt') as f2:
    for line1,line2 in zip(f1,f2):
    res.write("{0} {1}\n".format(l ine1.rstrip(),l ine2.rstrip()))

    f2.close()
    res.close()
  • dwblas
    Recognized Expert Contributor
    • May 2008
    • 626

    #2
    An SQLite tutorial is at http://zetcode.com/db/sqlitepythontutorial/ It uses a smaller subset of SQL commands than officially available. But these commands will work for any SQL interface if you just want to create a database and maintain the records in it. Note that if you use another SQL (not SQLite), you have to have Python bindings for it installed. So if you use PostgreSQL for example, you would have to install one of the Python bindings from https://wiki.postgresql.org/wiki/Python

    Comment

    • manyatha
      New Member
      • May 2017
      • 3

      #3
      Thank you it's help full.but currently am getting data as file,so there I need to convert column names and data types I have done.but finally create statement unable to insert last few line.if any suggestions it should be appreciated.

      Comment

      Working...