Page 9 of 9

Re: Weather Underground Plugin

Posted: Thu Jan 31, 2019 9:21 pm
by george tirebuyer
Lame indeed. Looks like I need to update my skill set to HTML scraping

Re: Weather Underground Plugin

Posted: Fri Feb 01, 2019 6:01 am
by kgschlosser
It's pretty simple to do.
if you open a webpage and click on view source. you will see the source code for the webpage.. In terms of weather I did a google search for local forecast.. and it brought up a link to weather.com with all the proper search parameters filled in for where I live. I used that URL and added some options to the URL so you would be able to specify your location.

I then opened the url in my browser. and then right clicked on the page and clicked on "View Source code". this showed me the raw data.. I scanned the source code looking for the information I wanted... It happened to be that this one was really simple because everything is stored in a JSON structure. so i needed to find a marker I could use at the start and at the end. where there is only a single occurrence of the start marker in the source. ctrl + F told me that. now that I had my start marker. i located the end of the structure which is a }; the brace and semicolon is not going to be in the structure at all. it will only appear at the end of it. so i did a find for the start marker.. then i did another find for the }; telling find to start at the position i got when i searched for the start. and that gave me the end of the structure.. fed it into the json library (json.loads()) and this spat out a python dict... and I know you know what that is.. from there just found the elements i wanted pulled them out and printed them


so enough of my babbling.. here is the code..

follow the instructions at the top of the code...

Code: Select all

# you need to set these to get the proper forecast
ZIP_CODE = '80439'
COUNTRY = 'US'
LOCALE = 'en-US'


import requests
import json

response = requests.get('https://weather.com/weather/today/l/{0}:4:{1}'.format(ZIP_CODE, COUNTRY))
content = response.content

start = content.find('window.__data={') + 14
end = content.find('};', start) + 1

data = json.loads(content[start:end])

location = data['dal']['Location']['locId:{0}:4:{1}:locale:{2}'.format(ZIP_CODE, COUNTRY, LOCALE)]
hourly_forecast = data['dal']['HourlyForecast']
pollen_forecast = data['dal']['PollenForecast']
observation = data['dal']['Observation']
daily_forecast = data['dal']['DailyForecast']


print '*' * 40
print('LOCATION')
print '*' * 40
print json.dumps(location, indent=4)
print
print
print '*' * 40
print('HOURLY FORECAST')
print '*' * 40
print json.dumps(hourly_forecast, indent=4)
print
print
print '*' * 40
print('DAILY FORECAST')
print '*' * 40
print json.dumps(daily_forecast, indent=4)
print
print
print '*' * 40
print('POLLEN FORECAST')
print '*' * 40
print json.dumps(pollen_forecast, indent=4)
print
print
print '*' * 40
print('OBSERVATION')
print '*' * 40
print json.dumps(observation, indent=4)

Re: Weather Underground Plugin

Posted: Fri Feb 01, 2019 6:14 pm
by george tirebuyer
OK I've got the data for my area. Under observation is the temperature:
10:08:05 AM "altimeter": 30,
10:08:05 AM "temperature": 58,
10:08:05 AM "uvDescription": "Moderate",
I 'm trying to get that into a variable to use and can't figure it out.

Re: Weather Underground Plugin

Posted: Sat Feb 02, 2019 7:21 am
by kgschlosser

Code: Select all


print observation['altimeter']
print observation['temperature']
print observation['uvDescription']
and if the data is nested...

Code: Select all

print observation['layer1']['layer2']['data']
you would need to replace layer1, layer2 and data with the appropriate "keys"

Re: Weather Underground Plugin

Posted: Sat Feb 02, 2019 3:19 pm
by george tirebuyer
Thanks, The data was nested 4 deep.
print observation['geocode:32.82,-116.91:language:en-US:units:e']['data']['vt1observation']['temperature']
Works!

Re: Weather Underground Plugin

Posted: Sat Feb 02, 2019 9:52 pm
by kgschlosser
anything else I can help you with??

Re: Weather Underground Plugin

Posted: Wed Jul 03, 2019 7:18 am
by blaher
https://api.weather.com/v2/pws/observat ... xxxxxxxxxx


I have this returning a json output in my browser, with my key that i get for uploading data from my personal weather station. Are there any updated plugins, or suggestions how to scrape the data into eventghost, please?

Cheers!

Re: Weather Underground Plugin

Posted: Wed Jul 03, 2019 3:13 pm
by kgschlosser
I do not have an API key for weather underground. as it is now a pay service. If you need this plugin to work I can modify it but I would need the API key to be able to do it. Let me do some digging into it.

Re: Weather Underground Plugin

Posted: Fri Jan 24, 2020 9:04 pm
by loveleejohn
kg! You've done some amazing work on this plugin and I and many others are thankful for it. It really sucks that weather underground folded their free api service. For now Open Weather Map is working though. Sadly it can only work in 10 minute increments but I suppose that is safer in some ways.

Re: Weather Underground Plugin

Posted: Sat Jan 25, 2020 9:02 pm
by kgschlosser
This is what happens when something that was free and OpenSource gets bought out by a corporate conglomerate (The Weather Channel)

Re: Weather Underground Plugin

Posted: Sun Jan 26, 2020 12:12 am
by loveleejohn
kgschlosser wrote:
Sat Jan 25, 2020 9:02 pm
This is what happens when something that was free and OpenSource gets bought out by a corporate conglomerate (The Weather Channel)
:evil: Too true...