web-scraping-challenge
Mission To Mars:
Web-Scraping-Challenge:
Description
The task of this assignment, was to build a web application that scrapes various websites for data related to the Mission to Mars and display the information in a single HTML page.
There were three parts to this assignment:
Scraping
I completed my initial scraping using Jupyter Notebook, BeautifulSoup, Pandas, and Requests/Splinter. This had four parts: The Latest Mars News, Featured Mars Image, Mars Facts, And the Different Mars Hemispheres.
The Latest Mars News
Using the Mars News Site , I scraped and collected the latest News Title and Paragraph Text and put them into two variables news_title and news_par.
Featured Mars Image
- Using the Mars Space Images Site I used the Splinter module to navigate the site and find the image URL for the current Featured Mars Image, and assigned the URL string to a variable called
featured_image_urlby combining what was scraped and the original URL.
Mars Facts
Using the Mars Facts Site and Pandas, I was able to scrape the table containing facts about the Earth and Mars and converted the data to a HTML table string.
Mars Hemispheres
Using the Astrogeology Site , I obtained high-resolution images for each hemisphere of Mars by creating a For loop that captured both the image and the title. I stored them in the img_url and title variables.
MongoDb
I converted my Jupyter notebook into a Python script called scrape_mars.py by using a function called scrape. This script scraped all of the data and was stored into a dictionary called mars_data. I connected my MongoDb and created a database mongo database titled mars_app and a collection titled mars_data.
FlaskAppliication/HTML
I then created a root route / that queried my mongo database to pass the data into an HTML template that displayed my data, using a HTML file called index.html.
