Using Beautiful Soup – Data Wranging with MongoDB
Articles,  Blog

Using Beautiful Soup – Data Wranging with MongoDB


Okay, we’re in the home stretch here. Let’s figure out how to build the HTTP request such that we can pass all seven of these form elements. Now, the question is where do these elements come from? In most cases when you see form elements showing up that aren’t obviously part of the form itself, and by that I mean aren’t part of the user interface. Now what’s gotta be going on here is that we have hidden form elements. So, let’s figure out where they’re at. So, here’s the start of the form. If, we look down through, I don’t obviously see any other elements. So, I bet it’s in some of this stuff that’s zipped up here. So, let’s take a look. And, sure enough, right here we find three hidden form elements. So, now we’ve accounted for six of the seven elements we’ve got to pass along, the Carrier, the Airport, the Submit button and then these three. And we can see that right here, Viewstate okay, is one of those with a very long value. If I open this up, I see that here we have one of those other values. The other one in fact, that makes scrolling through all the form data in the network tab such a long and tedious process. Okay, so now we’ve identified where all seven of the form fields are. The question then becomes, how do we build our HTTP requests, such that this data is included? Because that’s really what we have to do here. In addition to the Carrier, the Airport, and the Submit button, we’ve gotta send these along as well. So in this exercise what I’d like you to do is write some code here that uses the Beautiful Soup Module, and assigns values to Event Validation and View State such that they have the correct values. The values that this data source is expecting to see come across. In a little bit, we’ll talk about why these values are necessary. For right now, let’s just focus on building the request properly.

Leave a Reply

Your email address will not be published. Required fields are marked *