Vellamo Open shares the heart of [[Vellamo>>http://vellamo.quicinc.com]] with the open source community. Our benchmarks quickly became widely accepted by the industry and they sometimes use clever techniques to get reliable measurements out of a very different range of devices.

You are encouraged to look at the code, learn from it, build it and try it yourself.

Release 1.0

Included in this release are 6 benchmarks from the HTML5/Browser Chapter and all the support code to make them work. This support code includes our instrumented webview(s), native code that can enable an internal web server on the fly (for network performance measurement), and various utility functions you may find useful.

Downloading and Building from Source

Make sure you have a development machine (for example with Ubuntu 14.04 LTS) with the latest Android SDKs installed.

Download the source code with:

$ git clone git://codeaurora.org/quic/vellamo/open vellamo-open

Then just fire up your Eclipse IDE and create a new Android project from the source code contained in the "vellamo-open/source/" directory, or use Android Studio to import the project and convert it to a Gradle project.

To build the JNI (native) portion of the Vellamo Opensource project:

$ cd vellamo-open/source/jni/
$ ./vellamo_build_native.sh

This will create all the native libraries that Vellamo needs.

Now refresh the project in your IDE and you will notice that the native libraries are found and added to the project. At this point you can build and deploy to your device.

Benchmarks

There are six benchmarks that are part of the Vellamo Opensource 1.0 release. The source code of the benchmarks can be found on: source/src/org/codeaurora/vellamo/benchmarks/html5/

Image Scroller

A benchmark that quantifies the user experience of the browser's image decode and rendering engine. In this test, a highly dense image is scrolled from top to bottom, by the browser. The image itself is heavily dithered so it stresses browsing system performance. The final score is determined by measuring the scrolling rate in fps with a penalty if any stuttering is found.

Inline Video

An HTML5 <video> element benchmark that test the performance and capability of the browser's core video support. The test first plays back a sample video that is streamed from Vellamo's onboard web server locally from and to the device. It then probes the device's ability to support the most popular base video codec and adaptive video streaming standards. A score is then calculated based on initial video startup time, video performance in frames per second, and the number of codecs and streaming standards supported.

Surf Wax Binder

Measures the throughput performance of the "bridge" bindings code that links the JavaScript virtual machine with the rest of the browser's core layout and rendering engines. The test executes a long series of nested function calls, which read and write data between the browser and virtual machine. Throughput is then measured by counting the number of data access operations per second.

Load And Reload

A browser benchmark that measures the HTTP and Internet Protocol networking and associated web page object cache performance. This benchmark also uses Vellamo's onboard web server from which a sample web page is loaded and re-loaded in a continuous manner. An initial run is completed with caches cleared, and a second iteration is then completed with caches unaltered. Vellamo runs the complete benchmark in two major iterations; first emulating the network latencies found on a typical WiFi network, and then again as found on a typical cellular 3G/4G network. Each iteration also benchmarks the performance of the object caches. The final result is then calculated by aggregating the normalized page loading time for the four passes and the efficiency of the network system of the device.

Ocean Scroller

A benchmark that quantifies the user experience when scrolling a typical real-world web page. Vellamo mimics an end user in using a virtual finger to "fling" a sample web page repeatedly, measuring performance of the scrolling rate, and also records stuttering or artifacts that were incorrectly introduced. The score is based on measuring the scrolling rate in fps with a penalty if any stuttering is found.

Ocean Zoomer

A user experience performance benchmark for the browser's zooming capabilities. Within the benchmark, Vellamo injects virtual gestures that then zoom in and out from a variety of different portions of a sample web page, mimicking a real-world user. The speed is then determined by the time required to execute each zoom in/out operation that was completed. The smoothness is determined by evaluating the fluidity of the operation. The score is a balanced mix of frames per second rendered and the speed and smoothness of the action.

Licenses

The source code available for download from Code Aurora may be covered by one or more different licenses.  The files in Code Aurora may contain changes and additions on top of the code from the original source.  These changes and additions are covered under the same license as the original source.  In many cases, the license is explicitly listed at the beginning of the file.  A list of licenses is included for reference purposes only.

Support

See the Vellamo benchmark on the Google Play Store

For more information on Vellamo Open: https://www.codeaurora.org/vellamo-open

For support: vellamo-support@quicinc.com

Thank you for choosing Vellamo.

Tags:
Created by Greg Stinocher on 2013/04/29 16:12
   
This wiki is licensed under a Creative Commons 2.0 license
XWiki Enterprise 7.4.3 - Documentation