Google I/O 2013 – Volley: Easy, Fast Networking for Android

Google I/O 2013 - Volley: Easy, Fast Networking for Android



hi I'm ficus Kirkpatrick I'm an engineer on the Android team I work on the Play Store and I'm here today to talk to you about something we made that makes it really easy to develop superfast networked applications for Android it's called all right yeah very good it's called Bali so what is Bali well it's a networking library so when I say volley you might be thinking of something like this you know a client sends a request to a server over the net it comes back metaphor complete message received but when I named it I was really thinking of something more like this mobile app design is getting better and better but also more and more demanding for engineers a lot of current designs are really beautiful with a lot of rich metadata and tons of images but these things require tons of network requests to get all those all that data and images and there's a tremendous opportunity to make our apps faster by running those requests in parallel so you probably don't want to do quite as many requests in parallel as shown up here I think if we stick to the metaphor that looks kind of like a denial of service attack but a volley does make it very easy for you to run your requests concurrently without having to really think about threading or networking yourself sorry threading or a synchronization yourself the library comes with most things you need right out of the box things like accessing Jason api's or loading images are trivial to do with volley but it's easy to grow with your needs as well you can do there's a lot of customization options you can do your own custom retry and back off algorithms you can customize request priority and ordering and a lot more it also includes a very powerful request tracing system that we've found invaluable in debugging and profiling our network activity on the Play Store okay so why do we even need a networking library at all Android already has apache httpclient it already has HTTP URL connection true it sure does people have made tons of great apps with them so here are four apps that I use all the time on my phone Foursquare Twitter YouTube and news and weather these apps are great I love them and they have a lot of things in common they display paginate idli of the items typically with a thumbnail for each item you can click on something and view an item with more metadata or more images and there's typically some kind of a write operation or post request type thing like posting a tweet or writing a restaurant review or something like that and all of these apps had to reinvent several wheels to do what are at heart basically the same tasks doing response caching so things are fast getting retry right executing requests in parallel everybody pretty much rolls their own them so what we wanted to do with volley was both provide an interface where you can step up a little bit and think a little less about wire protocols and network transports and more about logical operations as well as just wrap up a lot of the things that we've learned in developing production quality apps at Google over the last few years so volley was designed for exactly the kinds of needs that those applications have relatively small RPC like operations fetching some metadata or getting a profile picture and put them in the view hierarchy it's also perfectly fine for doing those same kinds of operations in the background but it really excels at the intersection of UI and the network from its it's not for everything though responses are delivered whole in memory so that makes for an API that's really easy to use but not a good one for streaming operations like downloading a video or an mp3 run that's okay you don't have to use it for everything there are several apps at Google that use volley for there are PCs and use another system Download Manager or whatever for a long long lived large file download operations okay so let's take a simplified example of that kind of common pattern that I talked about I built a sample app that shows a title and a description loaded page by page from a server with a nice-looking thumbnail image for each for each item okay so we have a server that speaks a relatively simple JSON protocol this is by the way going to be the basis of my micro blogging startup so if any of you are angel investors please come see me afterward the protocol is really simple we've got the outer response is a is just a list of items Aysen objects with an optional token to retrieve the next page and each item is simply a title description and an image URL all as strings pretty straightforward on the client side again we have a pretty typical application architecture we've got an activity it holds an adapter and a ListView the adapter loads data from the network from the API server and manufacturers views for the ListView upon request okay so here we are in get view in our adapter in a typical implementation the standard trick here is to use the position passed in to get view as a hint for when you should download the next page for example if you have 10 items in your page when the ListView asks you for the 8th thing you go ahead and call load more data to fetch the next page of data okay now how is load more data actually implemented the usual approach is to use an async task or something like that so here in doing background we open an HTTP URL connection we get the input stream for the response data we make a byte array output stream to copy it into we use this magical function that I've written a hundred times in my life to copy it from the input stream to the output stream and then we pass it to the JSON object constructor for parsing and return it back so pretty straightforward conceptually one thing to note here is that there's actually a ton of boilerplate crap I had to cut out in order to make this fit on one slide so you don't see all the extra async task stuff and I think more annoyingly you don't see all of the try catch blocks here so you have to try catch for i/o exception if there's a network problem or JSON exception if there's a parsing error and then you have to close your input stream and then closing your input stream can throw too so you need to try catch there and have a finally block and this whole thing is probably twice this size or more if you were to see it in something that would compile finally we land back in on post execute on the main thread with our parse JSON object we pluck out the individual items from that root JSON object append them to the list that backs our adapter call notify data set changed and then let the ListView call us back to actually make the views okay now we're back in get view with some data so now we can populate our two text views with the title and description strings and we now have the image URL so we can kick that off for loading and again if you're familiar with this you typically use an async task or something like that so let's look at a naive implementation of a image loading async task here in the constructor we squirrel away the image view that we want to populate from the image at the given URL and then in doing background it's pretty straightforward from there you open the HTTP or l connection you get the response stream and pass it off to bitmap factory for image decoding again I had to cut out all the try-catch you know muckety-muck boilerplate stuff it's all pretty straightforward so far but tedious finally again we get back in onto the main thread in on post execute and we just simply set image bitmap on our image view that we saved in the constructor okay so there are actually a whole bunch of problems with the code I just showed you things are anything from outright bugs to things that are just way too inefficient to ship so I want to talk about some of the higher-level issues and problems and then discuss volleys approach to solving them okay first the code as written all network requests execute in serial so one after the other and this is because you called async task execute and I'm blaming you for calling it even though I wrote the code then you didn't call async task executes on executor or maybe you didn't call you didn't pass a thread pool executor to async task or maybe you didn't even know you have to do that it also means that in order to fetch the next page of data if the user scrolling down you have to wait for all the image requests to complete because things are scheduled strictly first-in first-out volley automatically schedules your network requests across a thread pool you can choose the size of the pool or just use the default we have the best results with four threads in our testing in the Play Store so that's the default we chose volley also supports request prioritization so you can set your main metadata at a high or medium priority set your images to low and then if you even if you have even a single thread you never have to wait for more than one image request to complete before you can fetch that next page of data and let the user continue scrolling down if you rotate the screen your activity gets destroyed of course it gets recreated and you start over from scratch so that in this case that means reloading absolutely everything from the network so this isn't a big deal right you write your own cache you maybe put your images in a hash map or an LRU cache and of course you still have to do that which is tedious and it also doesn't help you for the next time the user launches your app and and you need to reload everything again because those caches are only going to live for the lifetime of your process volley provides transparent disk and memory caching of responses out-of-the-box so what is transparent caching mean it just means that the caller doesn't have to know about the existence of the cache when there's a cache hit it simply behaves like a superfast network response you can provide your own cache implementation or we have one that you can use in the toolbox right out of the box and it's very fast we typically see a response times of a few milliseconds for a typical you know 10 to 50 K average size JPEG image the volley also includes some more advanced tools particularly to image loading and I'll talk about that in a minute ok so those of you who have written an app like that this probably saw the bug in my loading load image async task right off the bat you we've got a bunch of async tasks in flight and they're all loading the images and maybe the user is scrolling around like crazy and the views are getting recycled in the ListView so by the time the async task actually completes onpostexecute runs and we set the bitmap in an image view that doesn't really belong to the async task anymore ok so what do you do here you have a view holder and you maybe keep the URL in it and make sure that it matches but then you're letting a request complete which it doesn't matter anymore you're just going to throw the results away which makes things slow and it's wasteful so then maybe you keep track of your async tasks and canceled when you get recycled again more work volley provides a really powerful request cancellation API where you can easily cancel a single request or you can set blocks or scopes of requests to cancel or you can just use the network image view that I'm going to talk about in a second from the toolbox which does everything for you okay one last problem until Gingerbread HTTP URL connection was buggy and flaky so for those of you who would try to run this sample code on Froyo or before you might run into some problems here so you could use apache httpclient but that's also buggy and no longer supported so that's okay volley abstracts away the underlying network transport so in fact if you use our standard setup helper methods on Froyo you'll automatically get an Apache stack which works best there and on Gingerbread and above you automatically get an HTTP URL connection stack one one really nice thing about this approach is that it doesn't just solve the problem of this kind of varying behavior of these HTTP stacks but it also creates an opportunity so say you want to port two squares new okay HTTP library yeah I'm a fan you can you can just replace that in one place in your app without having to go do a complete refactor of your code it's like a much more targeted place okay so let's look at how we diplomat this app using volley there are two main things you interact with in volley two main classes request queue and request request queue is the interface you use for dispatching request to the network you can make a request queue on demand if you want but typically you'll instead create it early on at startup time and keep it around and use it as a singleton here we also make an image loader and I'll talk about that in a little bit okay here's something dense here in the volley version of load more data we make a JSON object request which is one of the Stan request that comes stock in the volley toolbox and we add it to the queue we provided with a listener and a listener is the standard callback interface you use for receiving your responses and the response listener here looks a whole lot like the async task version and onpostexecute we just parse out the json append the items to our backing list and call notify data set change so there are two bigger things I want to point out here one is this is it this is the whole thing I went on and on about the boiler plate before the boiler plate is gone this this is the whole thing in two n second the fact that this response listener looks a lot like onpostexecute in the async task version means that it's easy to port your app to volley so you don't have to boil the ocean and rewrite your app if you want to start taking advantage of this okay getting the image is even easier we just stow away the image request in our view holder and cancel it when we've been recycled when we're getting recycled an image loader again from the volley toolbox handles the rest it provides the memory caching for you you can even pass in drawables for loading or error States and an image loader will take it take it away from from there for you but maybe you're a programmer after my own heart which is to say super lazy you can just use network image view in the volley toolbox it's a subclass of image view you can simply replace your usages of image view in your layouts with this you call set image URL and you're done when it gets detached from the view hierarchy the request is automatically canceled if it's still in flight that's it one one subtle but I think pretty awesome thing about volleys image loading code is response batching so if you use image loader or network image view the library holds your image responses for a short period of time after after one comes and attempts to batch them together in a single pass through the main thread what this means in practice is that batches of images load in the UI simultaneously or for instance say you're running an animation when you load an image like you fade it in all those things fade in together in sync and just will secret sauce on it okay so the image stuff I think shows that volley handles the basics out of the box pretty pretty strongly but I also want to show what happens when you run out of the tool box and you need to roll your own we have a tool box but it doesn't include everything but it's really easy to extend so let's let's take a look at making a new type of request if you haven't heard of the jisun library it's a it's a json serialization and deserialization library that uses reflection to populate your java model objects from json objects you just make java classes that look like your json schema pass it all to JSON and it figures it out for you so here's the relevant snippet from a decent adapter for volley this whole thing is about 60 lines of code now this is kind of the meat of it you can see there's a gist URL on-screen you can look at the whole thing to drop into your app if you want but it's a you know it amounts to you know a few lines of code so we make a string out of the data that comes back from the server we pass that to Jason's from Jason method along with the class object of our route response that helps with reflection we use volleys HTTP header parser helpers from the toolbox and we're done that's it you can even see we handle those parse errors and that happens at the response time not necessarily in your application code where you're thinking more about errors and less about every different type of them that could happen okay let's take another look at load more data with that new cool request we just made so instead of using JSON object requests would make a new JSON request and pass in the class object of our list response and then that's it it's even smaller we append the items to our backing store and then call notify data set changed one cool little thing about this is that it means that the parsing actually happens on the background worker thread instead of in the main thread like when we do it on onpostexecute so you get a little bit more parallelism there for free okay I want to change gears a little bit and talk about the underlying implementation of volley kind of how it works some of the semantics it defines and give a little look under the hood as a implementer not necessarily the user so volley like I mentioned before handles operating a thread pool for you there's one cache dispatcher thread there's one or more network dispatcher threads this diagram shows the flow of a request through the dispatch pipeline the blue boxes are the main thread that's typically going to be your calling code and the response handler the green boxes are the cache dispatcher thread and the orange boxes are the network dispatcher threads the cache dispatchers job is essentially triage it figures out whether we can service a request entirely from cache or whether we have to go to the network to get it such as in the case of the cache miss or perhaps a cache hit but that's expired in the case of an expired cache hit it's also responsible for forwarding the the cache response to the network dispatchers so that they can be reused in the event of an e-tag match and the server has an opportunity to give you a 304 when we do end up needing to go to the network like in the case of like in the case as I mentioned the network dispatcher services the requests over HTTP handles parsing the responses using that parsing code I just showed and then post the response back to the main thread as a whole parsed object as a user of the library you typically don't have to think about any of this stuff you incur your request from the main thread you get your responses on the main thread and that's a but if you really want to dig in and learn more about the request pipeline volley has a firehose of debugging information that's available for you if you if you want it you can just set a system property and and you get verbose output of the complete lifetime of all stages of the request dispatch pipeline so here's one random chunk of log I took from my sample app when it was in the middle of loading a bunch of images at the top you can see that this request took 443 milliseconds it was low priority since an image and it was the 11th request processed by this dispatcher each subsequent line below that describes another step through the dispatch pipeline you can see on thread 1 that's the main thread at oh and by the way each each line shows the incremental time that that step took now on the main thread the request is added to the queue 68 milliseconds later it arrives at the cache dispatcher which means there was a little bit of contention for the cache the cache triage thread there the cache dispatcher figures out right away that it's a hit but it's expired and it forwards it along to the network dispatcher pool now it takes a hundred and thirty second 136 milliseconds to begin processing by the network dispatcher which is even longer so there's actually even a bit more contention for the network dispatchers than for the cache dispatcher you can see it takes the HTTP request takes 127 milliseconds now this is an expired cache it for an image so one thing to note is that this would typically be much faster and you'd get that 304 but this is a homemade webserver I made and it's crappy and doesn't have any features so don't do that and finally parsing the request takes about a hundred milliseconds in this case it's an image requests so parsing means image decoding one side note 100 milliseconds is actually a pretty long time to decode an image what's actually going on here is there is a whole flurry of requests going on and our image decoder doesn't allow more concurrent image decodes than there are CPU cores which is one more thing that we found to be optimal in our testing and development on the Playstore lastly we write the response bytes to the cache from the network dispatcher and we post the parsed object back to the main thread and we're done so this is a pretty advanced feature and I don't think most people will use it but we have really found it invaluable in digging in and profiling our use in the Play Store here's one common thing that you can find using this log so if you're seeing a large amount of time between post response and done what that means is that there's a lot of contention for the main thread if there's a lot of contention for the main thread you're doing too much on the main thread you can dig in from there okay so why do I keep going on about the main thread obviously you can't touch the UI from a background thread so you'd have to post back there anyway but my obsession with the main thread and ask anyone who works with me it is an obsession is about more than just the inconvenience of having to post a runnable back when you do everything on one thread and don't share any state with your workers synchronization is implicit you don't have to think about locking you don't want to think about locking because locking is hard and I'll give you an example how many people have ever written a block of code that looks like this do you okay me too so here's what happens here we are our async task is completed we ran on post execute and something has crashed we're touching the UI the UI is dead but it's way too late in the schedule to figure out why this is happening so we cram a little null pointer check in our code it doesn't crash anymore and we head off to the launch party but it's a waste of effort it's a waste of CPU cycles it's a waste of network bandwidth and and it's waste a battery to allow our request to finish that that you're just going to ignore the result of so it also leaves these these warts of like if null checks at the top of all your methods all over your code I mentioned volley has a powerful cancellation API and the main thread interface is with it like this volley always delivers responses on the main thread what that means is that once canceled returns you can be guaranteed that your response will not be delivered so here you can just keep track of all the requests you have in flight and then cancel them a non-stop you know that when onstop returns and you're no longer allowed to touch the UI the request won't be delivered so you don't have to check for null at the top of all of your listeners but you can actually do this with async tasks too so how about this I mentioned volley lets you set up blocks or scopes of requests it lets you tag any request with an arbitrary object that you can use to set up a bolt cancellation scope so in this example you tagged all the requests with the activity they belong to and then you simply pass those to cancel all in on stop and you can be guaranteed that you're not going to get any more responses you don't have to use the activity you could define a smaller scope like say you want to group all thumbnail requests from one view pager tab cancel them when you swipe away you can do that too if you really want to go nuts you can actually specify a custom filter deciding which requests to keep and which requests to cancel say for example you've got a post request in the background and you want to let that go but you want to cancel all your thumbnails you can do that too okay so we've covered how easy it is to build an app with volley or even port your app to volley and how to grow up from there just on the porting topic I want to mention those of you who have the Google i/o 2013 app that was ported to volley about a month ago and the whole thing took about half a day and it wasn't me who did it by the way so you don't have to be an expert um we've again seen those complicated cases defining custom requests doing you know more advanced cancellation operations and you can do it with a lot less code and a lot more functionality than you could enroll in your own it's easier than doing it yourself and you get the benefit of all of those all those lessons we've learned the hard way in developing our apps at Google but it's also faster than doing it yourself the Google+ team did a benchmark last year of a bunch of different networking libraries and volley one every single one they ran on Froyo through Ice Cream Sandwich and jelly bean on different types of hardware they ran on edge 3G 4G Wi-Fi volley was faster every single time in some cases by as much as a factor of 10 okay so you're sold how do you get started it's pretty easy just clone the project we have the repository URL right up here on the screen from there it's pretty simple there's an eclipse project I guess we're going to need to add an Android studio one right there in the repository there's also a build XML file so you can use ant to make yourself a jar or you can just drop the code into your app any of those things you call volley new request q start adding requests to the queue and you're off and running that's pretty much it anything else yeah that's all I have thank you very much for coming I I did want to leave time for QA and I will also be at office hours after this or you can hit me up on G+ or Twitter so we can take some questions hi my name is Alexey beside y'all fusion can you talk about the size of the in-memory cache for bitmaps disk size of the cache for bitmaps and if you hit the disk cache on which thread does the bitmap decoding happens that's a great question so the sizes of the caches are completely configurable by you you can choose the in-memory cache size and the disk cache size separately all blocking i/o is done on background threads we never do blocking i/o on the main thread but it's actually really important to be able to do the in-memory cache stuff on the main thread before you ever even get to the HTTP stack the reason for that being that you don't want to leave back into the main looper again you want to know immediately that you have your bitmap so that you don't defer let a layout pass happen and then set your image bitmap and you get a little flicker so that is your question oh he asked about disk thread versus networking thread caching caching it from the disk is all read on the cache thread so there's serialization at the cache thread basically if things can't end with the cast thread and they can get stuck if one thing is reading from the cache we have a design for how to you know split up cache workers or defer the work somewhere else we just haven't needed it it's been fast enough hey so my name is Mike from Epic Systems I'm really happy that you guys were able to encapsulate the caching logic we had troubles trying to do that ourselves my question is one thing that we need to do is establish different links for cache objects so for a certain object we determine we deem it as static versus dynamic and we cache it for different lengths of time so that's something that's going to be supported by this volley structure yeah so he's asking about cache TTL basically so Bali's cache implementation on disk respects all the standard HTTP cache headers so if your server sets you know this thing expires immediately it respects that I don't think it doesn't even get written to cache if you say this thing lasts for a day this thing lasts for a year again expiration is respected we don't have there's no TTL in the in-memory cache and our experience things just thrash through way too fast but again you can pass in your own implementation if you need to high park is some Chuck from aweber communications and thank you for this I've seen a lot of different approaches to solving this problem and written a few myself and one one common thing that seems to occur a lot is the dreaded out of memory exceptions and particularly when you're dealing with larger images and lower end devices and so I'm wondering if you saw how volley compared to other solutions in the number of occurrences of this and some tips for dealing with that scenario yeah we definitely struggled with out of memory errors a lot it's a tough problem because if you have a lot of memory to use you can make things fast the solution that we ended up on that worked really well for us was a couple things one I mentioned that we don't decode more things concurrently than there are CPU cores that actually ends up having a pretty positive effect on your kind of instantly measured heat pressure and the other thing we do is that we actually in our in the Play Store we specify the size of the in-memory cache as a function of the screen size which kind of makes sense right because you typically want to have as much in cash to fill you know three screens worth of data and as it turns out app devices are typically built with the memory to support the screen that they need to fill so it tends to work out somewhat coincidentally so I'd say that like this isn't really a volley thing I think it's how to choose your cache size thing and so we don't use soft references or weak references they don't work well we use like hard references and set a strong budget and we we are conservative in setting that budget by scaling down with the screen and really with the number of pixels Thanks hi my name is Mark I would like to know when is this going to make it into the SDK manager and the second part of the question is lots of images small size the sorts a lot a lot like the network protocol speedy are you planning on integrating speedy into a volley first question first when is it going to be in the SDK manager this is easy to answer I don't know it's a it's easy enough to get clone I don't think we have a great answer for that so I will duck that and ask your and answer your question about speedy I'm really excited about speedy volley was it was on my mind we're designing it somebody right now is working on putting the ok HTTP library that squared just released you may have seen it as an underlying transport stack for volley I haven't I haven't tried ok with speedy yet but according to Jesse the author you can plug speedy into the backend of ok HTTP which would pretty much give you speedy and volley so yes you can do it it doesn't work yet but we intend for it to work really soon that's great hey I'm one of the things that we have found is that a lot of our activities tend to have a mapping to like an API call and what we would like to do is basically make the API call before we do the transition to the activity a lot of these requests can actually be cached or shouldn't be cached is there any way to serialize your request objects yes you can kind of roll your own or the really dirty thing that we've done before is you make a separate request queue with one worker thread and and then things are first-in first-out if they're the same priority so you can do it it's the API is not optimized for it ok any plans to optimize the API for Dorada as a future is just no god traditionally actually what we do in the case where we have a API calls that need to be made in the serial is we initiate the second one from the response handler of the first one oh sorry bye serialize the API to call I actually mean like par set so that you can pass it across the activity boundary um I see so what I would do in that case I think is just passed the URL of your request or the the metadata in a parcel bowl or something right right but then the request isn't going to start until after you make the transition across activities ah I see um so I think your option there is gonna be to kind of shove in the static somewhere yeah yeah I don't think there's a great answer to your question okay Thanks hi my name is jonathan just have a quick question and all your examples you were using JSON requests and responses I was wondering if there was any way to use different types of protocol like XML for example or sure uh yeah I think I just forgot to mention it um we use protocol buffers pervasively in the Play Store and the Play Store runs on volley so yeah you can use XML you can use G as use Jason because it's easier to read on the slides but yeah we use protobuf you can use XML we have some things that are done as raw strings obviously images are all just formats of requests okay so it should be pretty easy to plug in whatever you want to use very much and if you look at the gist I showed in the presentation can I give you a starting point of writing your own like XML particular request okay sounds good and last question is there anything built-in to endow with retries in case of yes yes it does handle retry for you I didn't have time to cover it right now but you can set a custom retry policy you can set back off algorithm all that stuff all right sounds great thanks a lot umm so I'm gonna hold questions for now I'll be in office hours if you guys want to talk to me in person thanks again for coming please scan the QR code to rate this session unless you didn't like it and and thanks very much you

7 Comments

Leave a Reply

Your email address will not be published. Required fields are marked *