At New Relic, we’ve always strived to make the lives of our customers easier and today we’ve raised that bar even further. We’re excited to announce the release of the New Relic Platform – a seamless new way to monitor the performance of any technology through our intuitive and easy-to-use interface.

With the New Relic Platform, you’ll be able to get in-depth, real time insight into the performance of each component of your application stack – from cloud services, to the database, caching, networking, queuing and more. And best of all? You’ll be able to do it from a single pane of glass to monitor all the metrics you care about. No more logging in and out of different tools to manage the components in your app environment.

Today’s release includes more than 50 plugins available on the New Relic Platform:

Plugins are free to any of our users, and offer built in alerting capabilities and free storage for all monitoring data. Don’t see the plugin you’re looking for? Use your expertise to build one yourself in less than a day to monitor your metrics while leveraging the New Relic Platform, SDK and data visualizations.

To find out everything the New Relic Platform has to offer, visit newrelic.com/platform. You’ll quickly get insight into all the metrics you care about and wonder how you ever got by without it.

The post Measure All the Things! Introducing the New Relic Platform appeared first on New Relic blog.

But don’t just take our word for it. See the videos and quotes below to see how some of our customers are already getting value out of the New Relic Platform:

TicketFly:

By plugging all of these technologies into the New Relic Platform, we’re able to make breakthroughs in solving our problems much faster. It’s like having an MRI of our applications, showing us a slice of data across all systems and making it easier for us to draw correlations.
– Matthew Baird – VP of Engineering, Ticketfly

BeenVerified:

A lot of companies want to keep their product tight. But with this platform, New Relic shows that it wants to remain open and let users flesh things out. We know first-hand that people find ways to use our product in ways we couldn’t possibly predict. The key is to embrace that, promote it, and support it – and that’s exactly what New Relic is doing.
– Jason Amster – CTO, BeenVerified

MeetMe:

I would still be flying blind without New Relic. I could get my job done, but frankly it wouldn’t be as easy and it wouldn’t be as fun. It’s truly an enjoyable thing for me to dig through application data, discover issues and fix them. New Relic makes it possible for me to spend much more of my day doing precisely that.”
– Eric Peterson – Drupal Web Developer, Tableau

We no longer need to run our own custom metric tracking for all of our resources because we can now get all of that information inside the New Relic Platform. We can aggregate analytics in a single app. We don’t waste time looking for data – it’s all in one dashboard. And with each new plugin, we have a new opportunity to save a little more time.”
– James Carr – Systems Engineer, Zapier

Thank you to TicketFly, MeetMe, BeenVerified, Zapier, Tableau and all the other companies that worked with us through this launch. We couldn’t have done it without you!

Get Started with the New Relic Platform Today!
Find out how you can get started with the New Relic Platform today. And check out our case studies to see what some of our other customers are saying about the New Relic Platform.

The post Making our Customers Lives a Little Easier with the New Relic Platform appeared first on New Relic blog.

Introducing Platform Plugins
The New Relic Platform addresses this challenge with the new plugin architecture. Plugins provide a way to monitor each of these technologies, extending the New Relic interface with custom-made dashboards specific to each. They pair the reporting of metrics specific to the technology being monitored with a first class visualization of those metrics.

Plugins dramatically expand the possibilities for what you can monitor with New Relic, in a manner that retains all the New Relic goodness you’ve come to expect. Now you’ll have immediate, actionable insight in an easy to use UI for all the technologies you rely on everyday.

A wide variety of plugins are available through New Relic’s Plugin Central, so take a look and see what it has to offer. And if you don’t see one that you need, consider creating one yourself. It’s easy and fun to do.

A Little More About Plugins
Plugins are made up of two parts:

* An agent that observes the system being monitored and reports metrics about that system up to the New Relic data center

* A matching set of dashboards that display those metrics

Any type of technology – from a database to a caching system or switch – can have an agent written specifically for it. And such an agent will generate metrics that make sense for that technology. Additionally, each type of agent is paired with a UI specific to that agent. So it not only shows those technology specific metrics, it also arranges the information in a way that makes sense to anyone who uses it. For example, the arrangement of dashboards for a database will make sense for those who manage that kind of database and would be significantly different for those of a caching system.

A subset of those metrics is shown in a summary page for a specific plugin and New Relic monitors those metrics continuously. Users can set thresholds for specific metrics based on predefined limits. If a threshold is passed, an alert can be sent via email or any other notification mechanism New Relic supports.

Anything that can provide performance metrics can have a plugin written for it. The only requirement is that it has some kind of API to gather the metrics. Plugins work by polling that API, gathering up the metrics and reporting them up to a New Relic account.

Installing and Running Plugins
There are generally two types of plugins: those written against the Platform SDK that have an on-premise agent running and those written to integrate with a SaaS provider.

For the SDK, the language choices are Java and Ruby. While the install process can vary from plugin to plugin, it’s usually quite simple:

1. Specify the license key for the account that will receive the metrics.

2. Configure the plugin with information about the system it’s monitoring (e.g., the database servers)

3. Run the agent

* For Java, run a single .jar file
* For Ruby, do a bundle install and bundle exec of the gem

Because plugins monitor the systems externally through an API, they have minimal impact on the systems being monitored. For example, if a plugin stopped working for some reason, the system would not be affected. SaaS plugins do not need to be installed and only require a license key for sharing.

Soon after a plugin starts reporting metrics to your New Relic account, you’ll see the plugin appear in the left hand navigation panel in New Relic. (It will also be labeled with the word ‘New’.) From there, you can immediately start exploring the dashboards for that plugin.

Plugin Central
The New Relic Platform is open and extensible. This encourages our community to create a rich variety of plugins. As such, we have plugins written by our partners, our customers and ourselves. There may be multiple plugins for the same technology and all public plugins are available through Plugin Central in New Relic. And each publisher is responsible for supporting and maintaining their plugins.

Developing Plugins
The New Relic Platform provides an SDK that makes it easy to write on-prem plugin agents. It takes care of things like communication with the New Relic data center, running the polling loop, and logging. Plugin authors only need to focus on gathering metrics and calling a simple method to report their relative values.

Our goal is to ensure that the effort required to write a plugin is relative to the effort required to access that technology’s API. If it’s easy to get metrics from it, it will be easy to write an agent for it. From there, the UI is authored using an extension of the New Relic Custom Dashboards feature.

Currently you can choose one of two languages to write your plugins: Java and Ruby. We recommend Java because it typically has the simplest install. You can develop your plugin in whatever IDE you choose and manage it in any repository. (We recommend GitHub.)

In the future, we hope to be able to provide better development and user features by pairing Java and GitHub.

From Agent to UI
So far we’ve said that a plugin is comprised of an agent that reports metrics and a matching UI in New Relic. But how do we know which UI goes with which agent?

Each agent has a specified Globally Unique Identifier (GUID) within it. This is chosen by the plugin author and identifies the plugin forever. We recommend that you construct it from a reverse DNS of your company (i.e. com.mycompany.myplugin).

After your basic agent is written, the next step is to run your agent and start reporting metrics to your account. As we described earlier, you’ll soon see a new entry in New Relic. Its name will be the last segment of your GUID, and an asterisk ‘*’ will appear at the end to signify it’s new and under construction.

From there, you can select the plugin and begin authoring dashboards and specifying summary metrics. These are constructed from the reported metrics.

And that’s it!

Sharing your Plugin
You can create a plugin for your own in-house use. Or you may decide to make it available for the entire New Relic community. The decision is yours.

Publishing the plugin makes it available on Plugin Central for all New Relic account holders to see. It’s a simple matter of filling out some information, such as a description of your plugin and your support links.

Publishing associates the previously chosen GUID with that plugin. When other New Relic users run your agent, we can look up the UI you built and published for the plugin, and instantiate that UI in their account. As they start reporting metrics with your agent to their account, your plugin appears in their left side navigation. And clicking on it shows them their metrics in the UI you published. How cool is that?

Getting Started with the New Relic Platform
The New Relic Platform is designed to make your lives easier and more productive by bringing metrics from other tools into New Relic and expanding your view into key performance metrics across your technology stack.  It’s fun and easy.  Get started today by visiting our Platform page. And check out what some of our customers are already saying about building and using plugins in our case studies.

The post An In-Depth Look at the New Relic Platform appeared first on New Relic blog.

With the New Relic Platform, we’ve tried to make it easy for you to get the metrics you need from our easy-to-use, intuitive UI. But we couldn’t have succeeded without the support of the New Relic community and our plugin publishers.

In the videos below, our plugin publishers talk about the value the New Relic Platform offers their customers and their organizations:

MongoHQ:

Twilio:

Pivotal:

SendGrid:

EnterpriseDB:

Get Started with the New Relic Platform
Find the full list of available plugins on New Relic’s Plugin Central. And if you don’t see what you’re looking for, try building one yourself. It’s easy to do and easy to implement. For more information, see the New Relic Platform page.

The post The Metrics You Want, When You Need Them: Getting the Most Out of Your New Relic Platform Plugins appeared first on New Relic blog.

With that in mind, I asked attendees at the Velocity 2013 pre-party, “What are you doing differently this year to improve application performance that you didn’t do last year?” This is what they had to say:

The post Live from Velocity 2013: 22 Ways to Improve Application Performance appeared first on New Relic blog.

In this video, Steve explains why companies need to make web performance a high priority and how to bring that mindset to your organization.

The post Live from Velocity: Steve Souders on Fostering a Performance Culture appeared first on New Relic blog.

Once again, we’ll kick off the event with an ‘Optimized Breakfast‘ on Tuesday morning (June 18) with our COO Chris Cook, where you can hear company news and meet the New Relic team. Sign up here — seats are limited. If you’re not an early riser, then come network with us at our happy hour from 6:30-9pm Wednesday in our room at the convention center!

Then, join us for Office Hours on Wednesday. Meet New Relic’s Senior Director of Product Management Patrick Lightbody and Software Development Lead Bill Kayser at 1:15pm in Exhibit Hall 1. This is a great opportunity to talk with them about visualization techniques. Sign up now to make sure you get in!

And don’t miss our presentations! Here’s the lineup of what we have in store for you.

* Tuesday, June 18 – ‘Choose Your Weapon: A Survey for Different Visualizations of Performance Data‘ with New Relic’s Bill Kayser and Patrick Lightbody
* Thursday, June 20 (10:30am) – ‘It’s 10pm. Do you know where your code is?‘ with Patrick Lightbody.
* Thursday, June 20 (2pm) — ‘Surveying the RUM Landscape‘ with Patrick Lightbody (New Relic), Ilya Grigorik (Google), Buddy Brewer (SOASTA), and Jon Fox (Torbit).

For a full program list, see the Velocity site.

It’s not too late to get your ticket – join us at the Hyatt Regency in Santa Clara, CA. We’ll be at booth #309. We look forward to seeing you there! Come get your Nerd Life t-shirt and don’t forget: conference participants can get our New Relic Pro free for 30 days.

The post Get Up to Speed at Velocity appeared first on New Relic blog.

It’s super easy to get started:

* Install New Relic using your standard build tools and processes
* Debug and release your app in Eclipse just like always
* Build your Maven app with New Relic – it’s now as simple as adding a dependency to your application’s pom.xml
* Launch the emulator and see your New Relic data
* Enjoy New Relic support for your shiny new application!

You’ll find separate installation instructions for Ant, Eclipse, Maven and Gradle in our docs section.  And don’t miss our previous blog post on getting started with Gradle. For an example of how to install Eclipse, see this install video:

Don’t wait — log in, deploy and see data like you never have before on New Relic’s mobile section.

The post Android Developer? We’ve Got a Special Treat For You appeared first on New Relic blog.

“When we design apps, one of our top priorities is to make the experience fun, fast and simple,” says Matt Vlasach, Director of Mobile Integration Services at Unwired Revolution. “There’s no better way to encourage widespread enterprise adoption than to make an app genuinely enjoyable to use.”

Helping Clients Collaborate More Effictively
For one client — a major player in the food and beverage industry — Unwired Revolution designed DocLink, a software solution for accessing enterprise content on iOS devices and Windows 7 desktops. With this app, salespeople and other mobile employees can access files and auto-generated reports stored on enterprise servers through a common and very secure RESTful API. Users can subscribe to content they find interesting and that content is then automatically synced to their mobile devices for offline availability.

Another client — a major utility — needed an app to make mobile device management (MDM) a more user-friendly process. Unwired Revolution made it easier for the company to manage devices and maintain security posture without exposing thousands of end users to a complicated ERP backend. “In both of these engagements, we wanted to enable seamless functionality for enterprise users in an attractive, intuitive UI,” says Vlasach. “The ultimate goal was to unleash more productivity by transforming difficult, inefficient processes into engaging work experiences.”

Solving Their Own Performance Issues
But an app can only engage users if it performs as expected. During test phases for client projects, Unwired Revolution often had its own share of performance hiccups. “At one point, a pilot end user reported an increase in ‘server errors,’ but we couldn’t identify the source of the problem or reproduce it,” says Vlasach. “Our only option was to review log files and debug traces, which is hardly an efficient use of developers’ time. Besides, we didn’t want to burden our clients with the task of troubleshooting. Our goal is to be proactive — to identify problems before our end users even know that something’s wrong.”

Vlasach was already familiar with New Relic before bringing it to Unwired Revolution. In fact, he’d been using the solution for apps in his side business for nearly a year. “From my own experience, I knew that New Relic could help us address our app performance issues and give us early warning of issues,” he says. “We installed it on our production servers in February 2013, and right away it gave us deeper insight into any errors we encountered going forward.”

Server Side & Mobile Monitoring
Server Monitoring was just the beginning. Vlasch and his team began using New Relic for Mobile Apps as soon as it became available. It immediately gave them greater insight into their mobile app performance. “We got the Mobile iOS SDK up and running in literally five minutes, and we were getting valuable information right away,” he says. “It was amazingly simple: just insert two lines of code, add a framework, then run the app … With New Relic for Mobile Apps, we can clearly see which calls are being made, which ones are failing, how fast the servers are responding, and so on. There’s so much comfort in having instant visibility into our app’s performance — just knowing that everything’s OK.”

New Relic enables Unwired Revolution to identify and address performance issues on a proactive basis — long before those issues impact end users. This lets his team spend their time on something other than troubleshooting. We can spend more time doing something productive rather than chasing down a problem,” he says. “The investment we’re making in New Relic is trivial compared to the time it takes for a team of skilled developers to search for the source of a performance issue. From that perspective alone, the software easily pays for itself.”

And most importantly, better performance means a better experience for Unwired Revolution’s end users. “Our DocLink application is a big hit,” says Vlasach. “Users are saying, ‘This app is fun to use. It makes my work so much easier.’ That’s exactly what we’re going for — and performance plays a huge role in making their experience more enjoyable. At this point, I honestly can’t imagine releasing a mobile app without help from New Relic.”

Learn More
Read the full case study to find out how Unwired Revolution gained full visibility into its enterprise environments with New Relic.

The post Case Study: Unwired Revolution Gains Full Visibility into Enterprise Environments with New Relic appeared first on New Relic blog.

Existing tools weren’t a good fit. Our actual profile kept getting lost in the noise of tools like Instruments.app. We quickly realized we needed something extremely lightweight. And since our Mobile SDK runs on many different types of devices, we knew we needed a tool that could be distributed easily across a device test lab (such as the one our friends at AppThwack have built).

With those goals in mind, we developed dPerf – a small on-device profiling framework. Metrics collected by the profiler are pushed to a Node.js service with JSON over REST and are stored in a MongoDB collection. The Node service then provides a view that renders the profile data with Google Charts.

And here’s the best part: New Relic is releasing dPerf as an open source project! It can be found on the dPerf New Relic Repository on Github. Read on to learn more about how it works.

The New Relic iOS SDK
Before we dive into the technical details of dPerf, let’s talk a little bit about the New Relic iOS SDK.

Our approach to application monitoring has two main phases: instrumentation and reporting. The instrumentation phase collects data about network activities, such as NSURLConnection delegate method calls, into a memory buffer. Then once per minute, the reporting phase sends the contents of the memory buffer to the New Relic server.

It’s important to capture profiles for both phases of execution. This is a pretty easy task. (The profiler just needs a sane clock and to run for more than one minute.) If we know the start times for the application, profiler, agent and agent reporting phase, we can easily identify samples from the different phases.

Profiling Strategy
The general strategy for profiling a system is universal: sample the system at regular intervals. Any overhead incurred by the sampling process itself should be calculated and corrected by obtaining an average profile for the sampling process over a large number of runs, and subtracting this time from the samples.

The sampling interval is a critical parameter for a sampling process. Sampling theory (specifically, the Nyquist–Shannon sampling theorem) states that we can only accurately sample a process operating at a rate of R by sampling it at a minimum rate of 2R. This minimum rate, 2R, is known as the Nyquist rate. For example, to produce an accurate profile of a system performing an operation once per second, we must sample it twice per second. Sampling at a lower rate misses detail and introduces statistical noise. A good description of the Nyquist rate as it applies to audio sampling can be found here.

It’s ideal to conduct the samples as fast as possible, but higher sampling rates require more system resources. And this defeats the purpose of a lightweight profiling system. A critical task then becomes how to identify the ideal sampling rate to capture all relevant information, while minimizing the impact of the sampling process.

We’ve tested varying sampling rates between 10 – 100 samples per second. And we’ve found that a rate of between 10 – 50 samples per second is enough for most apps that do a lot of network activity. You can find that sampling at a higher rate reveals some interesting ‘jitters’ from external processes such as push notifications, SMS and the like.

In this project, the sampled value is the total CPU time used by the process across all threads, which is an ever-increasing value. By computing the changes between values, we can establish the relative CPU used between sampling periods.

What About Instruments.app?
Instruments.app is an amazing application. For the majority of cases, it’s more than adequate for profiling applications as its memory allocation tracking and reference cycle graphs are great tools for memory profiling. Its value is much more limited for CPU profiling, especially on-device.

Instruments.app is great if you want to aggregate the time spent in a particular method, context switch and system call. Unfortunately, it’s also resource intensive. The performance of method calls will still be accurate relative to one another, but the profile isn’t indicative at all of application performance on-device. Additionally, some handy tools such as the Dispatch instrument are not available when you attach to a process on-device.

Also, Instruments.app doesn’t make it easy to export raw metric data. Let’s say you’re trying to analyze dozens of profiled runs across dozens of devices. You want a simple and automated solution to compare the results.

On-Device Profiling Client
The profiler client has two main components: the periodic timer and the value function that’s being sampled.

Creating Fast Timers
iOS provides a few options for creating periodic timers. The NSTimer class is the most straightforward way to create a timer. NSTimer works reasonably well for timers with long intervals, such as a task that runs each second or more. For sub-second timers, NSTimer starts to become less than ideal. According to the documentation from Apple, NSTimer resolution is limited to around 50 – 100ms, but timers operating on this magnitude can experience a large amount of variability in the time that they actually fire.

The better choice for high resolution timing is to use Grand Central Dispatch (GCD) source timers. These timers are extremely accurate and add a fairly small amount of system load even at intervals around 10ms (100 samples per second). Dispatch source timers also fire at more regular intervals and have less variability than NSTimers. If the application doesn’t require precise timing, you can specify a ‘leeway’ argument when you create dispatch source timers to allow for more variability. For example, a timer can be created with an interval of one minute and leeway of five seconds. In this case, the timer will fire every 60 +/- 5 seconds.

Apple’s Concurrency Programming Guide provides an example of creating dispatch source timers:

dispatch_source_t CreateDispatchTimer(uint64_t interval,
                                      uint64_t leeway,
                                      dispatch_queue_t queue,
                                      dispatch_block_t block)
{
    dispatch_source_t timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER,
                                                     0, 0, queue);
    if (timer)
    {
        dispatch_source_set_timer(timer, dispatch_walltime(NULL, 0), interval, leeway);
        dispatch_source_set_event_handler(timer, block);
        dispatch_resume(timer);
    }
    return timer;
}

Using this timer is straightforward:

- (void) start
{
    _dispatchTimer = CreateDispatchTimer((1.0 / _sampleRate) * NSEC_PER_SEC, 0, dispatch_get_main_queue(), ^{
        [self sample];
    });
}

This method will create a dispatch timer which fires at _sampleRate times per second. We want it to be as accurate as possible, so a leeway of zero is given. It operates on the main queue and invokes a block that invokes the ‘sample’ method.

Sampling CPU Usage
Now that we have a fast and reliable timer, we can turn our attention toward interesting values to sample. We’re currently interested in application speed, so we need a good source for CPU utilization data.

iOS provides a great deal of low level information through the mach kernel task API, which is defined in mach/task.h. Apple has little documentation for these functions on iOS, but analogues exist for other mach-based kernels such as XNU and Darwin. These functions behave in more or less the same way across mach kernels, so documentation and code examples using this API are highly portable to iOS.

To find the aggregate CPU utilization for the application, we perform the following actions:

1. Call mach_task_self to obtain the current task.

2. Call task_info to retrieve basic info about the task including CPU time from terminated threads.

3. Call task_threads to retrieve an array of active threads.

4. For each thread, call thread_info which returns accumulated CPU time, and add to total.

Since these are low level kernel interfaces, take care when you allocate and de-allocate memory for the various structures.

The method below is a slightly modified version of the method provided in this Stack Overflow answer:

- (double)cpuTime
{
    task_t task;
    kern_return_t error;
    mach_msg_type_number_t count;
    thread_array_t thread_table;
    thread_basic_info_t thi;
    thread_basic_info_data_t thi_data;
    unsigned table_size;
    unsigned table_array_size;
    struct task_basic_info ti;
    double total_time;

    task = mach_task_self();
    count = TASK_BASIC_INFO_COUNT;
    error = task_info(task, TASK_BASIC_INFO, (task_info_t)&ti, &count);
    if (error != KERN_SUCCESS) {
        return -1;
    }
    {
        unsigned i;

        //
        // the following times are for threads which have already terminated and gone away.
        //
        total_time = ti.user_time.seconds + ti.user_time.microseconds * 1e-6;
        total_time += ti.system_time.seconds + ti.system_time.microseconds * 1e-6;

        error = task_threads(task, &thread_table, &table_size);

        //
        // failed to retrieve thread list: we can't proceed any further.
        //
        if (error != KERN_SUCCESS) {
            error = mach_port_deallocate(mach_task_self(), task);
            assert(error == KERN_SUCCESS);
            return -1;
        }

        thi = &thi_data;
        table_array_size = table_size * sizeof(thread_array_t);

        //
        // for each active thread, add up thread time
        //
        for (i = 0; i < table_size; ++i) {
            count = THREAD_BASIC_INFO_COUNT;
            error = thread_info(thread_table[i], THREAD_BASIC_INFO, (thread_info_t)thi, &count);

            //
            // if the thread_info call fails, clean up and fail hard.
            // partial results are probably useless.
            //
            if (error != KERN_SUCCESS) {
                for (; i < table_size; ++i) {
                    error = mach_port_deallocate(mach_task_self(), thread_table[i]);
                    assert(error == KERN_SUCCESS);
                }

                error = vm_deallocate(mach_task_self(), (vm_offset_t)thread_table, table_array_size);
                assert(error == KERN_SUCCESS);

                error = mach_port_deallocate(mach_task_self(), task);
                assert(error == KERN_SUCCESS);

                return -1;
            }

            //
            // otherwise, accumulate & continue.
            //
            if ((thi->flags & TH_FLAGS_IDLE) == 0) {
                total_time += thi->user_time.seconds + thi->user_time.microseconds * 1e-6;
                total_time += thi->system_time.seconds + thi->system_time.microseconds * 1e-6;
            }

            error = mach_port_deallocate(mach_task_self(), thread_table[i]);
            assert(error == KERN_SUCCESS);
        }

        //
        // deallocate the thread table.
        //
        error = vm_deallocate(mach_task_self(), (vm_offset_t)thread_table, table_array_size);
        assert(error == KERN_SUCCESS);
    }
    if (task != mach_task_self()) {
        error = mach_port_deallocate(mach_task_self(), task);
        assert(error == KERN_SUCCESS);
    }
    return total_time;
}

Who Profiles the Profiler?
There is some unavoidable overhead incurred by adding a profiler to an application that can be corrected for with benchmarking. The average time it takes to perform profiling operations is calculated with repeated trials and then used to correct sampled data. This technique should be familiar to many low level and embedded programmers.

The strategy is simple:

1. Record a timestamp at the start of benchmarking.

2. Perform N trials of the benchmarked operation. This should be a tight loop that does nothing but execute the operation.

3. Record a timestamp at the end of benchmarking.

4. Calculate elapsed time by subtracting the start timestamp from the end timestamp.

5. Divide this elapsed time by N (number of trials) to obtain an average time per operation.

A generalized method for benchmarking could be written such as:

- (uint64_t) benchmarkWithTrials: (uint) trials andBlock: (dispatch_block_t) block
{
    // call the block first, warming up any method call table lookups
    block();

    uint64_t start = mach_absolute_time();
    for(int t = 0; t < trials; t++) {
        block();
    }
    return (mach_absolute_time() - start) / trials;
}

Fortunately, the function calls to retrieve the task and thread info are very fast. On an iPhone 5, gathering CPU information for all threads takes on average about 150 microseconds.

Profiling Server
At this point, we have a tool that can record CPU usage efficiently at regular intervals. What should we do with this data?

We could have printed sample values to a console log for analysis or even written a small UI to display the profile after the profiling session was over. A goal of this project was to enable profiling across a variety of devices efficiently and scraping dozens of log files isn’t a scalable solution.

Instead, we wrote a basic client / server architecture. The profiling client buffers sampled data until the profiling session is completed. The data is then translated to JSON and sent to a RESTful Node service over HTTP. Along with the sample data, the JSON structure includes information about the device, a unique identifier for the profiling session, sampling rate and the name of the test performed.

This allows multiple devices to run a variety of profiling tests while reporting back to the same server. Decoupling the system in this way provides a lot of scalability and flexibility in the distribution and timing of tests.

Not including static content, the Node server is made up of only about 70 lines of code. It has two jobs:

* Store incoming data into a MongoDB collection.

* Serve that data up using Google Charts.

The server handles data posts from profilers, as well as generating charts and data sets for analysis. The following modules are used to accomplish this:

* Mongodb: Node driver for MongoDB

* Express:  Amazing web framework for REST and MVC mapping

* Jade: Powerful templating for dynamic views

* stylus and nib: CSS preprocessor

* Google Charts: Google JavaScript library for easy chart generation

This article is a great tutorial for using some of these modules to build a web app.

Distributed Testing with AppThwack
One of the major challenges of mobile development is ensuring compatibility with the wide variety of devices available on the market. iOS currently runs on 16 different Apple hardware platforms and Android runs on several hundred. Trying to build a test lab to cover all these permutations or perform tests on the various devices would be both cost prohibitive and time consuming.

AppThwack is a young company with an innovative approach to mobile testing. They have an automated testing platform that deploys applications to hundreds of real devices, runs tests and returns all sorts of interesting information. Along with reporting test failures, AppThwack also gathers screenshots and log files from all the devices.

With the AppThwack platform, we were able to deploy dPerf to multiple devices easily. When each device finished its test, it sent the samples to our central test server.

Results
After all of the pieces were in place, it was time to run some tests. To establish a baseline, we first tested an application that was completely idle except for running the New Relic agent and Profiler. Technically, we also ran it without the New Relic agent to get a true baseline, but an empty chart is not very interesting.

This chart clearly shows the New Relic agent reporting after exactly one minute. It’s pretty fast, too!

When this same idle test was run on the AppThwack platform, the results were a a bit different. One testing option AppThwack provided is to send several hundred random events during the testing period. This caused the ‘idle’ moments in the application to appear more jittery than what we saw in first test, which essentially had no events being fired. We also saw some of the AppThwack initialization during application start and the beginning of the random event stream. The New Relic reporting call was still visible when its timer fired after one minute.

Next, we checked the behavior of an application generating a bit of network traffic. This test performed a HTTP request to the highly useful httpstat.us site every second. We began with a plain old HTTP 200 OK request.

We definitely saw a periodic process here, mixed in with the ‘noise’ from the random event stream. It’s interesting to note that even at this point in testing, the New Relic agent used so little CPU that it was indistinguishable from the other network traffic.

Ramping up the network traffic to 5 requests per second, we saw a familiar pattern. The chart looked essentially the same as the 1 request per second, but the periodic process was multiplied. The New Relic reporting call was somewhat visible here, as a ‘thicker’ line around 60 seconds.

Next we moved on to an application that used more CPU resources. The chart below shows an application that queries the U.S. Geological Survey website for a GeoJSON data set of all the 2.5 magnitude earthquakes that occurred in the last month. After the request was complete, the GeoJSON was decoded. This process repeated every five seconds. The JSON payload size was quite large — roughly 600 kilobytes — so this was also a good test for handling of large HTTP responses.

Here the periodic process was quite visible and by the ‘thickness’ of the spikes we saw that the parsing took longer to complete — about a second or so.

The final verdict was that the New Relic SDK added only a tiny bit of overhead — about 10 – 20ms on average. And this overhead was only detectable in an application that was mostly idle, which is a rare case indeed.

Source Code
Again, if you’re interested in trying out dPerf for yourself, you can find the source for the profiling system in the dPerf New Relic repository on GitHub. We hope you find it as useful as we did. Feel free to use and enhance this tool to suit your specific needs. Got some cool changes that you want to share? Send us a pull request!

The post Introducing dPerf – A Distributed Low Impact CPU Profiler for iOS appeared first on New Relic blog.