首页 > 编程语言 > 详细

python获取hive表时间格式最大分区

时间:2019-11-07 15:31:53      阅读:311      评论:0      收藏:0      [点我收藏+]
#获取表的最大分区
import boto3
from datetime import datetime,timedelta
def get_max_partition(db_name,table_name,partition_format=%Y-%m-%d):
    client=boto3.client(glue)
    yesterday=datetime.utcnow()
    max_partition=yesterday.strftime(partition_format)
    response = client.batch_get_partition(
        DatabaseName=db_name,
        TableName=table_name,
        PartitionsToGet=[
            {
                Values: [
                    max_partition,
                ]
            },
        ]
    )
    while response[Partitions]==[]:
        if partition_format in [%Y-%m-%d,%Y%m%d]:
            yesterday=yesterday-timedelta(days=1)
        elif partition_format in [%Y-%m,%Y%m]:
            if yesterday.month >1:
                yesterday = yesterday.replace(month=yesterday.month - 1)
            else:
                yesterday = yesterday.replace(year=yesterday.year - 1, month=12)
        max_partition=yesterday.strftime(partition_format)
        response = client.batch_get_partition(
            DatabaseName=db_name,
            TableName=table_name,
            PartitionsToGet=[
                {
                    Values: [
                        max_partition,
                    ]
                },
            ]
        )
    return response[Partitions][0][Values][0]

 

python获取hive表时间格式最大分区

原文:https://www.cnblogs.com/wangbin2188/p/11811878.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!