Use Spark Sql With Hive Metastore

Spark SQL Spark SQL is a module for working with structured data spark = SparkSession.builder.config(conf=conf).enableHiveSupport().getOrCreate() spark.sql("use database_name") spark.sql("select * from table_name where id in (3085,3086,3087) and dt > '2019-03-10'").toPandas() Hive Data warehouse software for reading, writing, and managing large datasets residing in distributed storage and queried using SQL syntax. Hive transforms SQL queries as Hive MapReduce job. Besides MapReduce, Hive can use Spark and Tez as execute engine.

linux命令行笔记

命令行日常系快捷键 如下的快捷方式非常有用,能够极大的提升你的工作效率: CTRL + A – 移动光标到行首 CTRL + E – 移动光标到行末 CTRL + U – 剪切光标前的内容 CTRL

Shell学习笔记

shebang #!/usr/bin/env bash 这一行表明,不管用户选择的是那种交互式shell,该脚本需要使用bash shell来运行。由于每种shell的语法大不相同,所以这句非