网站API数据和CSV文件

机智的小拍之那个城市的天气如何？

大家好我是小拍，这两天有点忙就一直没上来

有两件烦心事，首先是老板最（yi）近（zhi）看我不顺眼，也不知从哪个数据库里导出一大堆的CSV文件，要我把每个文件的表头拿掉，只留下数据部分。这上千个文件要是手动一个文件一个文件去处理得改到天荒地老！看来只能想想python程序的辙了。

再一个就是给女神的早晚请安邮件短信不太奏效，我怀疑她看都没看就直接删除啦。得另辟蹊径了，你看我要是在请安信息中加点有价值的内容情况会不会好点泥？

要不你帮我看看我的腹稿中不中？

今天的太阳7:15分升起，正如我对你不断升腾的思念；这太阳会在5:42分落下，日落之后，我的思念会随夜色漫延开来。

我站在东经116.39北纬39.91度对我挂念的人诉说倾慕与思恋。天空是晴朗的，一如你明朗的笑容，冬日里零下四度的寒意亦驱不走这笑容带给我的温暖...今天的气压是...风力...湿度...可见度...

怎么越来越像天气预报啦！我的个old sky呀，这也太难写啦！

好吧，真相是我发现了个网站可以提供二十万个城市的天气报告，就想每天早上把天气信息抓下来，发给我心仪的姑娘，提醒她天冷加衣什么的。本来想把这些信息美化一下才发给她的，现在发现有点难！

至于老板的那些CSV文件，用python处理很容易。在文件夹里循环打开每一个文件，用CSV模块的读对象把文件内容读出来，略去第一行，把其余数据保存到列表里。再用CSV模块的写对象把列表里的数据写到原文件上，自然就把原文件给覆盖了。用程序来做一下就改完了，就算成千上万个文件也不是事儿！

import csv
import pprint

# Reader object
# exampleFile = open('example.csv')
# exampleReader = csv.reader(exampleFile)

# #
# exampleData = list(exampleReader)
# pprint.pprint(exampleData)
# #
# print(exampleData[0][2])
# print(exampleData[1][1])
# print(exampleData[6][0])


# csv.reader: How do I return to the top of the file?
# exampleFile.seek(0)
# for row in exampleReader:
	# print('Row #'+ str(exampleReader.line_num) + str(row))
	# print('Row #', exampleReader.line_num, row)
# exampleFile.close()

# The Reader object can be looped over only once. To reread the CSV file,
# you must call csv.reader to create a Reader object.

# writer object
# outputFile = open('output.csv', 'w')
# windows: outputFile = open('output.csv', 'w', newline='')
# If you forget the newline='' keyword argument in open(),
# the CSV file will be double-spaced.

# outputWriter = csv.writer(outputFile)
# outputWriter.writerow(['spam', 'eggs', 'bacon', 'ham'])
# outputWriter.writerow(['Hello, world!', 'eggs', 'bacon', 'ham'])
# outputWriter.writerow([1, 2, 3.141592, 4])
# outputFile.close()

# The delimiter and lineterminator Keyword Arguments
# csvFile = open('example.tsv', 'w')
# csvWriter = csv.writer(csvFile, delimiter='\t', lineterminator='\n\n')
# csvWriter.writerow(['apples', 'oranges', 'grapes'])
# csvWriter.writerow(['eggs', 'bacon', 'ham'])
# csvWriter.writerow(['salad', 'salad', 'salad', 'salad', 'salad', 'salad', 'salad', 'salad'])
# csvFile.close()



# removeCsvHeader.py - Removes the header from all CSV files in the current
# working directory.

import csv
import shutil
import pathlib

# assign the path of source and destination folder
srcPath = '/Users/Smonkey/Documents/Python/PythonABC_Online/2-17CSV/removeCsvHeader/'
destPATH = '/Users/Smonkey/Documents/Python/PythonABC_Online/2-17CSV/headerRemoved/'
#
withHeaderPath = pathlib.Path(srcPath)
withoutHeaderPath = pathlib.Path(destPATH)

# copy original folder or files
if not withoutHeaderPath.exists():
    shutil.copytree(srcPath, destPATH)
else:
    for f in [x for x in withHeaderPath.iterdir() if x.is_file]:
        shutil.copy(str(f), destPATH)

# Loop through every file in the current working directory.
for csvFilename in withoutHeaderPath.iterdir():
#
    if not csvFilename.name.endswith('.csv'):
        continue # skip non-csv files
#
    # print('Removing header from ' + str(csvFilename )+ '...')
    print('Removing header from ' + csvFilename.name + '...')

    # Read the CSV file in (skipping first row).
    csvRows = []
    csvFileObj = open(csvFilename)
    readerObj = csv.reader(csvFileObj)
    for row in readerObj:
        if readerObj.line_num == 1:
            continue # skip first row
        csvRows.append(row)
    csvFileObj.close()
#
    # Write out the CSV file.
    csvFileObj = open(csvFilename, 'w')
    csvWriter = csv.writer(csvFileObj)
    for row in csvRows:
        csvWriter.writerow(row)
    csvFileObj.close()

获得天气预报的数据也不难，openweathermap网站就免费提供这些，只不过是JSON格式的。所以我们得引进JSON模块，它可以帮我们处理抓下来的数据。不只这个天气网站，象Facebook, Twitter, Yahoo, Google, Tumblr, Wikipedia, Flickr, Data.gov, Reddit, IMDb, Rotten Tomatoes, LinkedIn……这些大咖网站都提供JSON格式的数据界面（API）供开发者开发应用使用（一般只需要注册个账号已获取API key），而且使用套路基本差不多：
按照网站的开发者网页上规定的格式把链接字串准备好
向网站提出个性化数据提取申请（比如我指定要某个城市的天气信息），得到JSON数据后，在JSON模块的帮助下，把JSON数据转成python变量（字典型）
对获得的数据做进一步个性化设置（比如我只要天气信息里的湿度和气压）和分析处理
把分析结果处理结果按指定的格式进行输出

这里用到的CSV模块和JSON模块使用起来都非常简单，不用白不用哦。大咖网站免费提供的数据更是如此！

import json

# Reading JSON with the loads() function
# stringOfJsonData = '{"name": "Zophie", "isCat": true, "miceCaught": 0,"felineIQ": null}'
# # #
# jsonDataAsPythonValue = json.loads(stringOfJsonData)
# # #
# print(type(jsonDataAsPythonValue))
# print(jsonDataAsPythonValue)

# Writing JSON with the dumps() function
# pythonValue = {'isCat': True, 'miceCaught': 0,
# 			   'name': 'Zophie','felineIQ': None}
# # #
# stringOfJsonData = json.dumps(pythonValue)
# # #
# print(type(stringOfJsonData))
# print(stringOfJsonData)


# # reportWeather.py - print weather report.
#
import datetime
import json
import requests
#
APP_KEY = 'a6f45ebb01197c60d1a850c9e1ff05db'  # Obtain yours from: http://openweathermap.org/
#
#
def time_converter(time):

    converted_time = datetime.datetime.fromtimestamp(
        int(time)
    ).strftime('%I:%M %p')
    return converted_time

#
def url_builder_name(city_name):

    unit = 'metric'  # For Fahrenheit use imperial, for Celsius use metric, and the default is Kelvin.
    api = 'http://api.openweathermap.org/data/2.5/weather?q='     # Search for your city ID here: http://bulk.openweathermap.org/sample/city.list.json.gz

    full_api_url = api + city_name + '&lang=zh_cn' + '&units=' + unit + '&APPID=' + APP_KEY
    return full_api_url
#
#
def data_fetch(full_api_url):
#
    response = requests.get(full_api_url)
    try:
        response.raise_for_status()
    except Exception as exc:
        print('There was a problem: {}'.format(exc))

    return json.loads(response.text)
#
#
def data_organizer(raw_data):
#
    main = raw_data.get('main')
    sys = raw_data.get('sys')
    data = {
        'city': raw_data.get('name'),
        'country': sys.get('country'),
        'temp': main.get('temp'),
        'temp_max': main.get('temp_max'),
        'temp_min': main.get('temp_min'),
        'humidity': main.get('humidity'),
        'pressure': main.get('pressure'),
        'sky': raw_data['weather'][0]['main'],
        'sunrise': time_converter(sys.get('sunrise')),
        'sunset': time_converter(sys.get('sunset')),
        'wind': raw_data.get('wind').get('speed'),
        'wind_deg': raw_data.get('deg'),
        'dt': time_converter(raw_data.get('dt')),
        'cloudiness': raw_data.get('clouds').get('all'),
		'description': raw_data['weather'][0]['description']
    }
    return data
#
#
def data_output(data):
#
#     # °C
    data['m_symbol'] = '\u00b0' + 'C'
#
    s = '''
----------------------------------------------
    Current weather in: {city}, {country}:
    {temp}{m_symbol} {sky}
    Max: {temp_max}, Min: {temp_min}

    Wind Speed: {wind}, Degree: {wind_deg}
    Humidity: {humidity}
    Cloud: {cloudiness}
    Pressure: {pressure}
    Sunrise at: {sunrise}
    Sunset at: {sunset}
    Description: {description}

    Last update from the server: {dt}
----------------------------------------------'''
    print(s.format(**data))
#
#
city = input('Which city you want to check? ')
#
url = url_builder_name(city)
rawData = data_fetch(url)
prettyData = data_organizer(rawData)
data_output(prettyData)

 

csv文件的读取和写入
 

你想了解哪个城市的天气状况？

 

1 1 1 1 1 1 1 1 1 1 Rating 2.00 (3 Votes)