Gibraltar

MAGIC Spell Studios LLC at Rochester Institute of Technology (RIT) looks forward to the Intel University Games Showcase each year. This is an incredible opportunity to experience what the most talented students in nationally ranked programs (by the Princeton Review) are creating. The future of our industry is in great hands with the talented, passionate visionaries who showcase their work at this event. Choosing John Miller to represent RIT was an easy decision to make. We were first introduced to John last spring following his participation in the Imagine Cup finals. John has all of the skills necessary to succeed and be a leader in this industry. His work ethic, passion, and talent are impressive and he is a terrific example of the caliber of students that we can offer to game development companies.

The Team:

• John Miller – Creator, designer, and developer
• Danielle Carmi  – 2D/3D artists and animators
• Angela Muscariello – Sound designer
• Elena Mihajlov – Music composer

The Inspiration:

John Miller says: “’Gibraltar’ is the game that I wanted to play while sitting in the back of class during high school. I wanted a game that I could play with one hand while still listening to the lecture, and that would hold my attention but wouldn’t make me feel trapped into a 45-minute session. I wanted to play something that was fast, simple, and low commitment while still strategically deep and intellectually stimulating. I also wanted to bring back that experience of sitting at a computer with a friend and playing old games like ‘Advance Wars,’ ‘Lego Star Wars,’ and ‘Backyard Football.’ The idea of Gibraltar bounced around in my head for a year and eventually I started putting those ideas down on paper and in my senior year of high school, I started development on the game. The art style and some mechanics have evolved along the way but the essence of that quick, lightweight strategy game is still there in Gibraltar today.”

The Game:

“Gibraltar” is a quick, turn-based strategy game in which two players send armies of adorable robots into battle for control of the game board. The more territory the player controls on the board, the more they can move their robots on their turn. This means that players are free to craft their own strategies and game play is very fluid. A match of Gibraltar usually lasts between 5 and 10 minutes, so you can fit a match in anywhere. Gibraltar is meant to be played between two players, sitting at the same screen. That kind of head-to-head competitive experience is something John always enjoyed. The player has four cute robot types to choose from when setting up their army, each with its own strengths and weaknesses. You only get to spawn your army once, so choose its composition carefully! The synergy between the different pieces allows unique gameplay and endless strategies and is simple enough for everyone to pick up. Players can also use special abilities that can change the course of the game but they are expendable and cost action points to play. Gibraltar features a story mode to help introduce players to the game and includes a fun cast of characters. The game ships a built-in map editor for players to design their own maps and play them with their friends.

Arkology

The USC GamePipe Laboratory has participated in the Intel University Games Showcase since it was created.

The GamePipe Laboratory faculty looks at the games being built in its Advanced Games course and other games shown at its Showcase, and then they agree on which game is the best. That game is the one that goes to IUGS, and this year it is “Arkology.”

The Team:

Core members:

• Powen Yao – Team lead responsible for overall design, research, and production
• Thomas Kao – Engineer responsible for overall project, architecture, network
• Joey Hou – Engineer responsible for overall project, gameplay
• Tiffany Shen – Artist responsible for models, textures, effects
• Grace Guo – Developer responsible for user interface, user experience
• Qing Mao – Technical artist responsible for 3D art, tech art, effects
• Eric Huang – Gameplay engineer

Newly-joined members:

• Leo Sheng – Gameplay engineer
• Jian Zhang – AI engineer
• Subhayu Chakravorty – Gameplay engineer

Former members:

• Divyanshu Bhardwaj – Gameplay engineer
• Christine Gong – Gameplay engineer
• Yue Wang – Gameplay engineer
• Guangxi Jin – Gameplay engineer

The Inspiration:

Powen Yao says: “As a PhD student under Professor Michael Zyda, I've been studying artificial intelligence in games and recently in virtual reality interaction. Real-time strategy provides a great platform for research and demonstration of game AI. My recent foray into virtual reality interaction led to my belief that it has great potential that could be unlocked with a combination of novel interaction techniques supported by AI. My interests in the two areas led to ‘Arkology,’ a virtual reality real-time strategy game. Right now, there are many virtual reality games in the market featuring swordplay or gun fights, but we want to take the game in a different direction. Instead of simply using the player's controllers to represent a gun or a sword, we are examining ways for the player to better process information and to effectively interact with many game elements at once. In other words, to design a real-time strategy game specifically for virtual reality to provide players with a unique virtual reality experience.

The Game:

In “Arkology,” the player has been chosen as the commander of Ark, a massive space-faring arcology designed to preserve humanity's continued prosperity and survival. The player can control the game using simple and intuitive motion control. From the Operations Room in the heart of the Ark, the player must strategize, command, and lead his forces to preserve what may be the last of humanity. The game can be described as a real-time tabletop war game where players need to control their miniature game pieces to fight the opposing force. A player's goal is to achieve the mission objective ranging from defending a valuable target to annihilating the enemy force.

Thematically, we want our players to feel like military commanders making strategic decisions and seeing their plans come to life. We want the players to feel like generals in World War II movies drafting their invasion plans over the map of Europe. We want to let the players live the scenes from the movie “Ender's Game” where the commander's will and orders get carried out by his officers.

Our focus for this project is in exploring novel virtual reality interactions to best utilize the fact that players have access to a three-dimension world. We are developing a series of virtual gears that will help a player better command an army and survey the battlefield. Some examples of what we have or are working on:

• Adaptable controller for the player to quickly change the functionality for the situation at hand.
• Augmented vision goggle to let the player see or hide additional game stats and information.
• A utility-belt for players to store and access game elements.
• Customizable battlefield camera and screen for players to monitor the battlefield.

Development and Hardware:

Professor Zyda says, “Having hardware donated from Intel helps our program out enormously and gives our students access to hardware of the future!”

Everend

The University of Wisconsin (UW)–Stout has been sending students, faculty, and alumni to the GDC in San Francisco since 2011, through both grant-funded travel and class-based domestic study-abroad opportunities. In addition, UW–Stout has attended the IUGS and always left inspired to create games and someday be a participant, as well. UW–Stout is happy to be selected as a participant this year because of its consistent rankings in the Princeton Review. Everend is a game that represents everything that UW–Stout is about: bringing artists, designers, and programmers together to create imaginative virtual worlds, stories, and characters.

The Team

• Will Brereton—Programming lead
• Emily Dillhunt—Art lead
• Mitch Clayton—Design lead
• Daniel Craig—Programming and lighting
• Megan Daniels—Art and public relations
• Gabe Deyo—Programming and cameras
• Phoenix Hendricks—Programming and sound
• Alex Knutson—Rigging and animation
• Logan Larson—Programming and level design
• Zachary Pasterski—Artist and level design
• Hue Vang—Cinematic and concept artist
• Tyler Walvort—Programming generalist
• Dave Beck—Professor and executive producer

The Inspiration

The team says, “The game idea was very loose at first and closer to a Japanese role-playing game in style. Kaia, then our unnamed purple owl protagonist, lived underground with a village of burrowing owls when their world started to shake and crumble. It was Kaia’s job to figure out why. During a pitch presentation, a professor who was sitting in remarked, ‘Why not have the character be out of place? Have the cave be a strange place,’ and that comment colored the rest of our development and sent us in the direction you see today. Environmentally, the game is inspired by the massive caves of Hang Sun Doong in Vietnam, where the caverns are large enough that forests grow within. We wanted to capture the ambient, colorful, vast feeling we got from looking at pictures of caves like these instead of the cramped, dark, wet feeling usually conjured up by cave environments.”

The Game

Players explore a vast, ancient cave and overcome its many obstacles as Kaia, an adolescent owl. Everend is a single-player, exploration-driven 3D puzzle platformer. Players solve various problems and puzzles throughout the environment as they progress through various levels of the caves, trying to reach the surface. Kaia collects various items throughout the journey, which allows for increasingly complex problems and solutions. The team wanted to focus on the atmosphere and ambience of the game environment, so they kept the UI and mechanics simple and noninvasive to place the focus on exploration. Players can also try to piece together the story of what happened in the caves as they explore by studying the cave environment and clues left behind in the form of cave paintings. Everend is a short, self-contained experience, with a character and environment the team hopes will leave a lasting impression.

Development and Hardware

The team says, “Our development team was lucky to have access to a dedicated game design computer lab in the School of Art & Design at the UW–Stout. The lab regularly updates their technology every two years with new workstations, so we had the opportunity to work with HP workstations that had game industry-standard Intel processors, complimented by high-end graphics cards and more than enough RAM. In addition, we were able to use these machines in the playtesting and exhibition of the games, making it a great overall experience that was all built on the foundation of Intel processor technology.”

Intel® Math Kernel Library (Intel® MKL) provides a mechanism to capture the run time information of BLAS and LAPACK domain functions used in a target application. This mechanism is enabled by either setting the environment variable MKL_VERBOSE to 1 or calling the support function, mkl_verbose(1). When a user application is run under this configuration mode, Intel MKL outputs the function name, parameters and time taken to execute the corresponding function. Below is the MKL_VERBOSE output produced by a sample program invoking MKL BLAS functions: 

Refer to the related chapter of Intel MKL documentation for more details about MKL_VERBOSE: https://software.intel.com/en-us/node/528417

MKL_VERBOSE can produce large amounts  of output (in the order of hundreds of MB’s or even tens of GB’s), when running applications that heavily use MKL. And it can be difficult to understand the usage of MKL functions in the application. To alleviate this problem, we provide a Python script that summarizes the output produced by verbose mode and generates a report grouped by function names, parameter list, call count and execution time. This tool takes an MKL_VERBOSE log as input and produces a summary report at stdout. Below is an example output produced by the tool:

The tools is provided as an attachment to this article. Users can find the download link at the bottom of the page. After download, rename the file to have the .py suffix.

You face a constant balancing act between features and performance as you build your game. The GPU is the most obvious bottleneck you'll encounter as you add graphical effects and features to your game, but your game can also become CPU-bound.

In addition to the usual CPU loads from game logic, physics and artificial intelligence (AI) calculations, the graphical effects that make your game feel immersive can be CPU-intensive, typically making the bottleneck shift back and forth between the GPU and the CPU throughout development of the game itself.

Modern microprocessors have great single-core performance, but depend on multiple CPU cores to give better overall performance. To use all that available CPU compute power, applications run fastest when they're multithreaded so that code runs concurrently on all available CPU cores.

This video showcases Ashes of the Singularity*, a recent real-time strategy (RTS) game from Oxide Games and Stardock Entertainment. You'll see how it delivers excellent gameplay and performance on systems with more CPU cores.

Figure 1: Ashes of the Singularity* shows how a well-threaded game can get better frame rates on systems with more CPU cores.

By building a new engine and using Direct3D* 12, Oxide made it possible for Ashes of the Singularity to use all available processor cores. It runs great on a typical gaming system and scales up to run even better on systems with more cores. You can use these same techniques in your game to get the best performance from your CPU.

Direct3D* 12 eliminates bottlenecks and allows high performance

To get the best frame rates in Ashes of the Singularity, the Oxide team used Direct3D* version 12. Earlier versions of Direct3D run well, but have a few bottlenecks. In version 12, the API incorporated several changes that remove these bottlenecks that tend to slow down games: multiple objects are now simplified into pipeline state objects; and a smaller hardware abstraction layer minimizes the overhead in the API and makes it possible to remove resource barriers from graphics drivers.

It's possible to create commands from multiple threads in Direct3D 11. However, there's so much serialization required that games never got much speedup from multithreading with the older version. With the API changes in Direct3D 12, this fundamental limit doesn't exist anymore. Without that serialization, it's now practical for games to fill command lists from multiple threads and have much better overall threading.

By taking advantage of these changes to the API, Ashes of the Singularity runs best on Direct3D 12.

The Nitrous* Engine makes it possible

Oxide wanted to create a more complex RTS than any built before, with support for larger armies with more units and larger maps. To build this next-generation RTS game, the development team knew that they needed a new game engine; existing game engines couldn't support the unit counts or map sizes they wanted. They started from scratch to build the Nitrous Engine* to make Ashes of the Singularity possible.

Any new engine must first deliver high-performance rendering. With that in mind, the Nitrous Engine has well-tuned support for the latest graphics APIs, multiple GPUs, and async compute.

The game supports many units for each player, as well as large maps. Simulating the physics of this many in-game objects across a large terrain generates a large CPU load. More importantly, the AI workload is massive since the behavior of each unit needs to be simulated. There's also an emergent property from the large number of units the game supports. With more units, it becomes harder for the player to directly manage units. Oxide built a layered approach to AI where armies cooperate in sensible ways that use the relative strengths of each unit while paying attention to their relative weaknesses.

To make this kind of scale possible, Nitrous threaded their engine by breaking work up into small jobs. The job system is designed for flexibility, and the small jobs can be spread out among as many CPU cores as possible. Oxide carefully tuned the job scheduler for speed since most Intel® processors include Intel® Hyper-Threading Technology, the scheduler also looks for locality between jobs. Jobs that share cached data are scheduled on different logical cores of the same physical CPU core, which is an approach that yields the best performance and job throughput.

Regardless of approach, there will always be a bottleneck somewhere when you add complexity to a game.  As you develop your game, think about the relative CPU and GPU loads that you might expect. Understand how your game will work when the GPU is the bottleneck and consider how it will behave when the CPU is the bottleneck.

Intel® Core™ processors make it shine

Intel® Core™ processors can help make your game shine like Ashes of the Singularity. As you design and optimize your game, target mid-range processors and design for scalability up to the most powerful processors.

By using the techniques we describe here, Ashes of the Singularity runs great on the best-in-class Intel® Core™ i7-6950X processor Extreme Edition, which has 10 physical CPU cores and a large cache for the best overall performance. With the work divided into jobs and a great GPU, the game's frame rate increases with more CPU cores. On identical systems with varying numbers of CPU cores, the frame rate improves steadily up to the max of 10 physical cores.

The game also includes some massive maps. Since the player and unit count get so large, the AI burden for a fully-outfitted set of players becomes huge. After careful tuning, Ashes of the Singularity allows these maps only for systems with large numbers of CPU cores (six or more) through the job scheduler, which  automatically puts work on all available cores. This is a great approach for you to pursue: detect your system's core count with a function like GetsystemInfo() if you need to selectively enable features.

Scalable effects add some sparkle

Although this game is mostly focused on ever-faster frame rates with more cores, there was a little extra CPU room for some bonuses. With more cores, Ashes of the Singularity will automatically enable advanced particle effects on some units as well as temporal motion blur.

Figure 2: Advanced particle effects on two large dreadnought units look awesome.

The particle effects give added visual impact, but they don't affect gameplay.

Figure 3: Temporal motion blur adds realism to fast-moving units.

With temporal motion blur, fast-moving units are simulated across multiple frames and then combined in a blur, adding visual realism to these units.

Check it out, and then build your own awesome game!

Ashes of the Singularity delivers excellent performance that scales up with available CPU cores. This is done through using the latest Direct3D API which is multithread-capable, more efficient job partitioning and scheduling, and active detection of core availability to enable more complex features. More cores unlock a more complex gameplay with larger maps. The combination of temporal motion blur and enhanced particle effects give a great-looking game. And now the recently-released expansion Ashes of the Singularity: Escalation builds on these advantages, adding even larger maps, an improved UI, and better performance across different platforms.

We hope you are inspired to apply these design principles to your development project, and create an awesome game!

Graphics tuning? Start with Intel® Graphics Performance Analyzers

Graphics optimization is a large subject in its own right. To get started tuning your game, we recommend the Intel® Graphics Performance Analyzers. Check out these tools at https://software.intel.com/en-us/gpa.

Sole

Drexel is proud to be invited to IUGS since the beginning. It’s a great opportunity for Drexel students to have their work recognized on a national stage. Holding it at the premier game development industry conference makes it a great practical exercise in promoting their projects, as well as a strong networking opportunity, regardless of the competition outcome.

Drexel’s Digital Media program produces many student projects every year, so when they open their internal competition, they get student teams applying from different years (sophomores through PhD candidates), courses, and programs under the DIGM umbrella. They hold their own competition using a format similar to the actual event, modeling their judging process on the IUGS rules, and adding the overall quality of the presentation to the gameplay and visual quality categories. With energetic discussions among the faculty, they choose from their 6-8 participating teams the one team that will represent Drexel.

It’s an exciting process that becomes a goal for the students, especially after Drexel’s first-place win for gameplay last year. Opportunities like IUGS inspire students to stay focused on their projects as goals beyond just grades and portfolios.

The Team:

• Nabeel Ansari – Composer who is responsible for the game’s beautiful music and audio design
• Nina DeLucia and Vincent De Tommaso – Artists who work tirelessly to paint, model, and sculpt all of Sole’s art assets
• Thomas Sharpe – Creative director and programmer

The Inspiration:

The team says: “The creative process for conceptualizing ‘Sole’ has been one of the most challenging and rewarding journeys of our artistic careers. We believe video games are an incredibly powerful tool for capturing abstract emotions that are hard to put into words. So in approaching the original design of Sole, we started with a particular emotion and worked backwards to find what kinds of interactions would evoke that feeling. The game’s core mechanic and thematic content were inspired by the internal struggles we’re currently facing in trying to figure out who we are and where we’re going in our personal and creative lives. In many ways, Sole is an allegory for all that uncertainty we’re feeling working through our first major artistic endeavor.”

The Game:

Sole is an abstract, aesthetic-driven adventure game where you play as a tiny ball of light in a world shrouded in darkness. The game is a quiet journey through desolate environments where you’ll explore the remnants of great cities to uncover the history of an ancient civilization. Paint the land with light as you explore an abandoned world with a mysterious past.

Sole is a game about exploring a world without light. As you move through the environment and discover your surroundings, you’ll leave a permanent trail of light wherever you go. Free from combat or death, Sole invites players of all skill levels to explore at their own pace. With no explicit instructions, players are left to discover their objective as they soak up the somber ambiance.

Development and Hardware:

The team says: “Developing the game on Intel’s newest hardware has given us the opportunity to experiment with many visual effects we previously couldn’t achieve. As a result, we are now able to incorporate DirectX 11 shaders for our grass, add more props and details to the environment, and render the world with multiple post-processing effects. The feel of our game changed dramatically once we had access to hardware that was capable of powering the latest rendering technology.”

It’s been a busy time for Out of the Box Network Developers and we have lots of news and information to share with you. Read on for information about upcoming events in Santa Clara, Portland, Shannon, and Bangalore, and get recaps of recent meetups and devlabs. Learn about new articles on Intel® Developer Zone, meet new Innovators, and check out some developer projects on DevMesh. To top things off, we’ve added a new section to highlight upcoming Intel® Network Builders University classes and events.

Contents

• Upcoming Meetups and Devlabs.
  • Out of the Box Network Developers Santa Clara.
    • Cloud Networking Deep Dive.
      Thursday, April 6, 2017, 5:30 PM to 9:30.
  • Out of the Box Network Developers Portland.
    • Talk about NFV/SDN concepts and overview of ONOS.
      Tuesday, 28 March 2017, 5:00 PM to 10:00 PM
  • Out of the Box Network Developers, Bangalore.
    • SDN/NFV industry standard DevOps implementation, Benchmarking and Testing.
      Tuesday, March 7, 2017, 10:00 AM to 3:30 PM.
  • Out of the Box Network Developers, Ireland
    • Enabling Virtualization in SDN and NFV world on IA.
      Wednesday, March 29, 2017, 9:30 AM.
• New on Intel Developer Zone.
  • Analyzing Open vSwitch* with DPDK Bottlenecks Using Intel® VTune™ Amplifier
  • Build Your Own Traffic Generator – DPDK-in-a-Box.
  • SR-IOV and OVS-DPDK Hands-on Labs.
• Recap of Our January 2017 Meetups.
  • The SDN/NFV Developer Party on January 19th at SC-9 (Santa Clara)
  • Enabling Virtualization in SDN and NFV world on IA (Bangalore)
• People and Projects.
  • Congratulations to our Two New Intel Software Innovators.
  • Latest Developer Projects.
• Intel® Network Builders University.
  • Data Plane Development Kit Courses.

Upcoming Meetups and Devlabs

Out of the Box Network Developers Santa Clara

Cloud Networking Deep Dive
Thursday, April 6, 2017, 5:30 PM to 9:30

SC9-Auditorium, Garage B
2250 Mission College Blvd, Santa Clara, CA

Google’s software-defined global network, underlying network virtualization stack (Andromeda) and network services, along with open innovations provide the foundation for securely delivering a diversity of workloads and services on Google Cloud Platform (GCP). Google Cloud Virtual Private Cloud (VPC) provides you with a secure and flexible sandbox to run your cloud workloads. Global Load Balancing and Cloud CDN deliver global reach, scale and high availability and secure your apps against DDoS attacks. Cloud Interconnect enables seamless connectivity options for hybrid/multi-cloud app delivery. This talk provides an in-depth look at Google Cloud Networking, what's under the hood, the benefits it delivers along with real world GCP customer stories. Presented by Prajakta Joshi, who is a product manager at Google focused on delivering Cloud Networking products to scale, secure and simplify your Google Cloud Platform deployments. Prajakta previously served as Director of Product for ONOS, a Software-Defined Networking (SDN) Control Plane Platform, which she helped productize and launch in open source.

Out of the Box Network Developers Portland

Talk about NFV/SDN concepts and overview of ONOS
Tuesday, 28 March 2017, 5:00 PM to 10:00 PM

Intel Auditorium RA1
2501 NW 229th Ave, Hillsboro, OR

This is our first Meetup! We will talk generally about NFV, SDN, ONOS and our experiences when deploying, running it, favorite flavors of vendors, etc. 

7:00 - 7:30   Networking
7:30 - 8:00   Overview of NFV and SDN by Baltazar Ruiz
8:00 - 8:30   Overview of ONOS (Open Network Operating System) by Karla Saur
8:30 - 9:00  Q&A

About Karla Saur - Holds a PhD in Computer Science, currently works as a Research Scientist on Distributed Systems.

About Baltazar Ruiz - Over 15 years of experience as Sr. Network Engineer, currently works as an Application Engineer in Intel.

Out of the Box Network Developers, Bangalore

SDN/NFV industry standard DevOps implementation, Benchmarking and Testing
Tuesday, March 7, 2017, 10:00 AM to 3:30 PM

Increasingly, service providers are creating labs to try out their own SDN/NFV solutions in the lab and Indian Telcom Industry is playing a critical role in this effort. In this meet up key player in this market e.g. Intel, TCS, and Tech Mahindra will talk about the roles they are playing in this effort.

Out of the Box Network Developers, Ireland

Enabling Virtualization in SDN and NFV world on IA
Wednesday, March 29, 2017, 9:30 AM

Irish Aviation Authority Conference Centre
The Times Building, 11-12 D'Olier Street, Dublin 2

Networking has traditionally been entrusted to customized ASICs and vertically integrated software solutions on top of these customized boxes. Virtualizing network functions on commodity x86 based servers brings its own challenges. Sharing resources among virtual functions and added software layers of hypervisor, soft switches, etc. before the packet gets onto the Network Function are some of them. This meet up will focus on giving the attendee a flavor of some of the tools and technologies being to resolve some of these issues from the Industry and Intel. To be presented by Andrew Duignan, who is an Electronic Engineering graduate from University College Dublin, Ireland. He has worked as a software engineer in Motorola and now at Intel Corporation. He is in a Platform Applications Engineering role, supporting technologies such as DPDK and virtualization on Intel CPUs. He is based in the Intel Shannon site in Ireland.

Please register to confirm a spot and a chance to win a 50 euro Amazon gift card.

New on Intel Developer Zone

Analyzing Open vSwitch* with DPDK Bottlenecks Using Intel® VTune™ Amplifier

This article shows how we used Intel® VTune™ Amplifier to identify and fix an MMIO transaction performance bottleneck at the microarchitecture level in OVS-DPDK. By Bhanu Prakash Bodi Reddy and Antonio Fischetti.

Build Your Own Traffic Generator – DPDK-in-a-Box

Build your own DPDK-based traffic generator with a MinnowBoard Turbot or any Intel platform. OS is Ubuntu* 16.04 client with DPDK. Uses the TRex* realistic traffic generator. By M Jay (Muthurajan Jayakumar).

SR-IOV and OVS-DPDK Hands-on Labs

Automate setup of SR-IOV with DPDK in Linux* and configure an OVS-DPDK NFV use case inside nested VMs. Provision into a cluster or datacenter. Includes all the setup scripts used by Clayne to prepare for the hands-on labs at the December 8 Intel® Developer Zone NFV/DPDK Devlab. By Clayne Robison.

Visit our library on Intel Developer Zone to see all of our Networking articles and videos.

Recap of Our January 2017 Meetups

The SDN/NFV Developer Party on January 19th at SC-9 (Santa Clara)

Sixty-three developers trained from 40 companies. This party was a spotlight on project pitches from the Out Of The Box meetup community, ending with a project idea brainstorming session.

There were lightning talks from industry developers from Intel, CableLabs, Apple, Huawei, and ONOS on topics such as RDT in NFV, OPEN-O*, ONOS*, and more.

Three developers from the Out Of the Box community gave prepared talks on ongoing projects stemming from the NFV/DPDK DevLab in December 2016, and one new developer from the audience came up and talked about their project. These developers were given DPDK-in-a-Box dev kits in recognition of their work.

Enabling Virtualization in SDN and NFV world on IA (Bangalore)

One-hundred fifteen developers trained. The first meetup was planned and executed in Bangalore to engage developers from networking companies, CSPs, networking and telecom equipment manufacturers, system integrators, and OEMs that are focused on adopting NFV and SDN technologies.

This meetup was remarkably successful with an audience of 115 developers, architects, and network engineers ranging from 64 different companies, exceeding the original expectation of 50 developers.

People and Projects

Congratulations to our Two New Intel Software Innovators

• Anthony Chow (SDN/NFV developer and blogger 6000+ followers)
• Shivaram Mysore (Developer for Faucet, which is an open-source SDN controller)

Latest Developer Projects

• Subrata G: DPDK Adaptation of libc Socket Calls
• Shivaram Mysore: Deploying SDN Wired/Wireless Network
• Tharaneedharan Vilwanathan: PerfectStream: A DPDK-based Video Gateway

Intel® Network Builders University

Data Plane Development Kit Courses

The Intel® Network Builders University released a series of new courses this month in the Data Plane Development Kit (DPDK) program. The new material adds a variety of deep-dive technical content to the existing library. Expand your DPDK knowledge and skills by taking the following new courses:

Setting Up DPDK on Different Operating Systems: In this course, you will learn the process of installing DPDK on a variety of operating systems, the installation of DPDK from source code, and the topic of setting up Hugepages.

The DPDK Packet Framework: This course covers the DPDK packet framework and provides example pipelines and use cases.

The DPDK Sample Applications: This course provides a basic introduction to a few of the 40-plus DPDK sample applications available today.

Writing a Simple DPDK Forwarding Application: In this course, Intel Software Engineer, Ferruh Yigit, covers the topic of writing a simple DPDK forwarding application.

In addition to the new deep dive DPDK content, the Intel Network Builders University is releasing a new course on containers in the Management and Orchestration program titled "Container Orchestration with Kubernetes."* To access the material, simply log in or register on the Intel Network Builders University page.

Download Document PDF 1.35MB

The Mission

When top UK-based studio Creative Assembly* began their ambitious work on Halo Wars* 2, they wanted the game to run on a variety of settings supported by DirectX* 12, and to be playable up and down the hardware ladder—including advanced desktop PC configurations and laptops. While many of the optimizations also enhanced the game for high-end systems with discrete graphics cards, this white paper will explore the team’s efforts for Intel integrated graphics and multicore processing functions.

Extending a Franchise

First-person shooter Halo* is one of the most popular franchises in PC gaming history; it started in 2001 with Halo: Combat Evolved, one of the original Xbox* launch titles. By late 2015, Halo had generated over USD $5 billion in lifetime game and hardware sales. Development work for Halo Wars 2 was performed by Creative Assembly, veterans of Alien: Isolation* and Total War: Warhammer*. They were experienced at producing games for multiple platforms; but, with DirectX 12 still maturing, the team faced challenges with their engine, the DX 12 driver, multicore efficiency, and more.

Figure 1. Halo Wars* 2 is the latest addition to the Halo* universe.

Switching the game to an RTS title meant tracking more units and packing the interface with crucial statistics, while updating the “mini-map” constantly. Halo Wars 2 also introduces “Blitz” mode—a new, innovative and action-packed twist on RTS gameplay that combines card-based strategy with explosive combat. The new mode also streamlines most of the traditional RTS systems, such as base-building, skills development, and resource management.

Expanding the User Base

Michael Bailey, Lead Engine Programmer at Creative Assembly, was one of the principal developers involved.

“Given the nature of RTS games, unit counts and large-scale battles with plenty of VFX were always a big focus,” he explained. “This, combined with a deterministic networking model common to RTS games, put restrictions on how scalable we could be, so we pushed from the start to ensure we had a good base to make use of multiple CPU cores. Our primary goal here was to ensure the game ran on a wide range of hardware, and the Microsoft Surface* Pro 4 laptop was a natural to target, aiming for 30 fps and still looking great.”

Figure 2. Dazzling particle effects keep the screen display busy.

Excellence across a wide spectrum of hardware was key. There is a temptation to develop early versions of a game to initially fit the specs for a high-end desktop system, taking full advantage of 10 teraflops of GPU potential. But this makes later scaling the game to Xbox One* and Ultrabook™ devices tricky, as early design choices could limit optimization options. With top-end Ultrabooks and laptops nearing the Xbox One in power and performance, those devices are now considered mainstream. That means developers can no longer concentrate on either CPU or GPU optimizations—both are crucial.

Figure 3. Getting the game to play on Ultrabook™ devices and laptops was a key task.

Bailey had not specifically optimized previous games for Intel® HD graphics, but working directly with Intel allowed him to get up to speed quickly. Fortunately, the graphics systems used by Creative Assembly were designed to be scalable without hurting the overall look and feel. Intel engineers talked Bailey’s team through their tools, on site, to diagnose issues and measure performance for added features. As the DX 12 version became more mature, areas for improvement were pinpointed. This included testing the performance of the game across multicore machines to ensure acceptable decent scaling.

The Game is On

In the early stages of the project, the team encountered several issues:

• Instability on the target system
• Speed issues at low settings on a discrete GPU
• Severe I/O lag
• Tools failure on the Universal Windows Platform (UWP) and DirectX 12
• Driver Issues
• Corruption due to buffers being reused before completion

None of the problems turned out to be show-stoppers, but the early screenshots revealed that there was a lot of work to do.

Figure 4. Early screenshots revealed corruption on a wide scale, affecting terrain, units, and the “mini-map.”

Multicore Optimizations

• The team worked on several different areas to optimize the game’s multicore support:
• Improve algorithms
• Reduce memory allocations
• Streamline assets
• Perform low-level optimization
• Enhance parallelization

In multiplayer battles, there were challenges ensuring that the simulation—spread over multiple threads—remained deterministic. On the CPU side, the team concentrated on being able to split the simulation while ensuring determinism. In addition, specific to DirectX 12, they worked to minimize resource barriers and redundant work.

The team found that changing the order of something meant there was a chance that they could end up with each client diverging and having a different representation of the Halo Wars 2 world. If two or more clients disagreed on a checksum, for example, the result was a “desync” that caused a player to be kicked out. One cause was a race condition where the output is dependent on the sequence or timing of other uncontrollable events. The team soon learned what calculations could put be put onto other threads, when they could run, and when it was safe to do that.

They eventually reached a stage where the CPU side was efficiently running across multiple threads, well apart from the render thread. Their work on multicore optimization will be presented at the 2017 Game Developers Conference (see link, and list below.

GPU Optimizations and Diagnostics

The Creative Assembly team performed multiple GPU optimizations, working closely with Intel® Graphics Performance Analyzers (Intel® GPA). These are powerful, agile tools which enable game developers to utilize the full performance potential of their gaming platform, including Intel® Core™ processors and Intel® HD Graphics, as well as Intel® architecture-based tablets running the Android* operating system. Intel Graphics Performance Analyzers visualize performance data from the application, enabling developers to understand system-level and individual frame performance issues, as well as allowing “what-if” experiments to estimate potential performance gains from optimizations.

The Creative Assembly team used new features of Intel GPA Monitor to launch and profile the game in action. They also examined problematic frames with Intel GPA Frame Analyzer to identify graphics “hot spots” in a particular scene.

Intel GPA tools also assisted investigations into corrupt terrain tiles, which the team tracked down to an error with descriptor tables. In addition, they investigated performance bottlenecks and judged performance versus quality tradeoffs for code that controlled terrain tessellation, as well as texture resolution bandwidth bottlenecks.

Bailey and the team also used Microsoft GPUView and other internal game profilers. The team scaled down their graphical options in the following areas:

• Disabling Async Compute when it ended up using too many synchronization primitives to guard between multiple command lists. (This slowed things down on hardware where Async Compute isn’t natively supported.)
• Reducing the terrain tessellation amounts for quality versus performance scalability reasons; on smaller screens, the extra detail afforded by tessellation wasn’t worth the cost.
• Dropping the terrain compositing tiled resource map sizes down to match the resolution the game was running in. This helped to avoid compositing terrain textures at resolutions that were too high for the display.
• Correcting the pixel shader system. The team had assumed that as the pixel shader wasn’t outputting anything within their shadow-rendering Pipeline State Objects (PSOs), it would be stripped along with interpolators from the Vertex shader. Intel GPA revealed that wasn’t the case.
• Tweaking the small-object culling factor to account for resolution (both in the main scene and in the shadows where the shadow resolution was much smaller).
• Swapping the heavy shadow PCF filter kernel to use GatherCmp to massively reduce the cost of that shader with no visual difference.
• Removing redundant terrain composition layers (a bug was causing them to export all layers, even unnecessary ones).
• Implementing a dynamic particle-reduction system which prioritized battle visual effects and dropped less important “incidental” environmental effects, or heavier “high-end” effects such as particle lights.
• Players can identify units by what visual effects they saw, or what they look like against the terrain. The optimizations Bailey and his team implemented had to keep the display readable without decimating the frame and disabling certain effects. “If the scene gets too heavy, we start ramping down the particle effects,” he said.

They also ensure that the battle effects look amazing. When there were 20 explosions on top of each other, they started turning off environmental effects and the less important visual effects. “Those are based upon on the current particle system load,” he explained. “On the Microsoft Surface Pro 4, they’re probably dropped down a bit more than they would drop down on a higher-end system which can handle more of a load. You get the environmental effects staying on, but they’re not really a big part of the battle.”

Figure 5. Big battles contain multiple units, plenty of particle effects, and lots of information to process.

DirectX 12 Challenges

Creative Assembly’s technical base is separate from the Total War: Warhammer engine, and the graphics layer is brought over from Alien: Isolation. They wrote the DX 12 graphics layer, building on top of existing DX 11 support, to take advantage of newer features and ensure it was more optimal than DX 11.

When they started, the graphics layer was targeting DX 11 for PC, so planning for and moving to DX 12 exclusively gave them plenty of opportunities to maximize performance. Bailey discovered during development that they were “over-cautious” with fences and descriptor table invalidations to ensure correctness and stability. They studied several DX 12 talks at GDC 2016 and gathered more information from online sources, until they understood the best way to take advantage of it in their engine.

As the system became more mature and stable, they started to optimize to ensure they removed redundant descriptor settings and unnecessary resource barriers and fences. This, combined with render pass changes to track the resource dependencies better, meant that they were able to shift some of their command-list building passes, such as shadows, to run in parallel with other areas and make more use of multiple cores. The command lists they moved to run in parallel were heavy on the CPU side due to the number of draw calls. These passes increased the cost with the number of units fighting on-screen, so they benefitted greatly from this optimization during heavy battle sequences. Their DX 12 version soon became faster than DX 11 for CPU-side rendering, even single-threaded.

Figure 6. In this screenshot, the "mini-map" in the bottom right corner is fully functional.

The trade-off was that the DX 12 environment was very new: enabling DX 12 validation layers to get diagnostic warnings and errors was not yet mature. The documentation was incomplete, or inaccurate. When things went wrong, Bailey’s team wasn’t sure if it was their fault, or if the driver was to blame.

Eventually, they were developing only on DX 12 and advancing to a single low-level graphics API gave them more control over the performance trade-offs they could make. This allowed the team to target a large number of Windows* 10 devices with a single code-path. The beta released in January 2017 supports DirectX 12 down to feature level 11.0; the game’s terrain texture compositing code requires “Tiled Resources Tier 1” support.

Challenges of Evolving Software

Because there were so many new features they wanted to exploit in the game, the team had to continually update drivers, compilers, and Windows 10 versions. Fortunately, they were in direct contact with various hardware vendors, which meant they had access to beta drivers with specific fixes, as well as beta versions of their internal tools.

Over time, the drivers, tools, and validation layers became more mature, so when errors did occur they were usually from the game code and were easier to track down. The graphics testbed helped debug a wide set of DX 12 API functionality, from developing new features to working in a much more cut-down environment where they were in control of all the variables.

Figure 7. Cut-down environment for testing new features.

Advice for Developers

Reach Out to the Experts

Bailey advises developers to reach out to the Independent Hardware Vendors (IHVs) for key insights on problems. “They obviously want your game working well on their hardware, and they will give you help, as well as tools,” he said. “If you can get them to take an interest in your game, or you’re taking interest in their hardware, then it’s a mutually beneficial experience.” Part of the payoff is access to early fixes and improvements, and optimization feedback, which Bailey’s team used extensively.

“I don’t think we ever got conflicting feedback,” Bailey said. The vendors were helpful in identifying known issues, and, in some places, the game works slightly differently when it identifies hardware from Intel, AMD*, or NVIDIA*. “They've all got their own slight differences, but talking to them is how you find out what they’re good at,” Bailey said. Overall, the result was a faster game, especially on the lower-end hardware.

Create a Robust Test Lab

The Creative Assembly engine team has six people attacking problems and testing solutions in a variety of ways. They often borrow machines as needed and rely heavily on remote debugging to a second machine or Surface Pro 4 at their desk—this makes working on a touchscreen device much simpler. “I’m not sitting there trying to use the touchscreen to try and get it to fit on the screen,” Bailey said. “I can comfortably use my big desktop to remote debug.”

Microsoft owns the rights to the Halo franchise, so Creative Assembly had access to Microsoft's compact testing lab through that relationship. It was a gateway to all sorts of different varieties of hardware, some below the minimum specs. Being able to regularly test builds increased their coverage and ensured that Creative Assembly’s optimizations were thoroughly checked.

Share Knowledge

Bailey encourages developers to reach out and participate within the community, especially by attending events such as the Game Developers Conference (GDC). If that’s not possible, Bailey suggests “downloading the talks to catch up on them later.”

Previously, there were only a few DirectX 12 talks. This year, there will be more tips and tricks shared by experienced teams. “You want to share your successes as well,” Bailey said. “If you share your success, then people will come and talk to you about how they did something differently.”

It’s like a multi-threaded attack on a problem, but with people instead of processors. “When you share these things, you invite the conversation,” Bailey explained. “And inviting that conversation means you might find out a better way of doing something. Maybe your approach was pretty good already, but things can always get better and better.”

Additional Resources

Intel Graphics Performance Analyzers:
https://software.intel.com/en-us/gpa

Intel Multicore FAQ:
https://software.intel.com/en-us/articles/frequently-asked-questions-intel-multi-core-processor-architecture

Introduction to the Universal Windows Platform:
https://docs.microsoft.com/en-us/windows/uwp/get-started/universal-application-platform-guide

DirectX 12 Installer:
https://www.microsoft.com/en-us/download/details.aspx?id=35

Microsoft Surface Pro 4:
https://www.microsoftstore.com/store/msusa/en_US/pdp/productID.5072641000

CPU and GPU Optimization Talk at GDC 2017:
http://schedule.gdconf.com/session/threading-your-way-through-the-tricks-and-traps-of-making-a-dx12-sequel-to-halo-wars-presented-by-intel

Nullam id dolor id nibh ultricies vehicula ut id elit. Nullam quis risus eget urna mollis ornare vel eu leo. Donec id elit non mi porta gravida at eget metus. Aenean eu leo quam. Pellentesque ornare sem lacinia quam venenatis vestibulum.

Step 1: Project Requested

STAKEHOLDER submits ticket in YouTrack, or has contacted the BC team about the request.

PROJECT MANAGER creates a kickoff meeting.

Step 2: Kickoff Meeting & Master Tickets Created

Kickoff meeting is held with STAKEHOLDER, PROJECT MANAGER, EDITORIAL, and UX.

Meeting agenda:

• Outline clear goals and success metrics of the new project
• Determine all team members needed to be involved
• Walk STAKEHOLDERS new to our process through the onboarding presentation
• Define scope (if time, a sitemap is created during meeting)
• Define development requirements
• Deadline requirements
• Design requirements

If scope is not finalized during the kickoff meeting or the project is a larger initiative, a discovery meeting is created by UX.
 

After the kickoff meeting the PROJECT MANAGER...

• Creates a master ticket, a design ticket and a copy ticket based on meeting notes.
• Assigns the design ticket to UX.
• Assigns the copy ticket to EDITORIAL.
• Creates a rough schedule and adds to the master ticket.
• Ensures training meetings are setup for tools (if needed).
  • GatherContent (GC) - EDITORIAL trains
  • Youtrack - Online self serve training
  • Author Training - WEBOPS trains
  • Book Training - WEBOPS trains

Step 3: Sitemap Created

UX creates sitemap and adds links to design ticket.

UX informs the PROJECT MANAGER that the GC space can be created.

Step 4: GatherContent Created

EDITORIAL creates GatherContent project based on UX sitemap, adds links to copy ticket and assigns ticket to PROJECT MANAGER.

If any training is required, EDITORIAL arranges a GatherContent training session.

Step 5: Content Outline Created

STAKEHOLDER adds a content outline to GatherContent. This will help inform the wireframes and copy guidance.

STAKEHOLDERS provide all visual assets for DESIGN team to reference.

Example: [link to a GC space with an example?]

Step 6: Wireframes Created

UX creates wireframes based on the copy outline and adds links to design ticket. 

Step 7: Copy & Initial Design Comps Created

STAKEHOLDER adds copy to GatherContent and obtains all team and legal reviews or approvals. This copy should be created according to design layout, trademark and branding, and style guidelines.

Copy should be at 90% complete.

PROJECT MANAGER watches status of GC and alerts EDITORIAL when copy is ready for editor review.

AT THE SAME TIME...

DESIGN creates comps based on the wireframes and content outline. If there are any issues, DESIGN waits till the initial copy has been added to GC (for problematic pages). DESIGN updates the design ticket.

DESIGN presents comps to STAKEHOLDER for validation.

Step 8: Copy Review

EDITORIAL updates workflow status in GatherContent through all stages of editing, and then changes status to “Approved” when complete.

EDITORIAL informs design if any issues may be present.

EDITORIAL updates copy ticket with appropriate information and assigns to PROJECT MANAGER.

During the time EDITORIAL is reviewing final content, DESIGN should verify that no changes are needed to the design comps.

If excessive changes are required, DESIGN lets the PROJECT MANAGER know.

If design comps are good to go, DESIGN preps assets for upload and assigns design ticket to PROJECT MANAGER.

Step 9: Web Production Tickets Created

PROJECT MANAGER creates web production tickets. This ensures that we don't have a backlog of tickets waiting while content and design are being finalized.

PROJECT MANAGER informs DESIGN and EDITORIAL and requests they add assets and GC links to appropriate tickets.

PROJECT MANAGER assigns tickets to WEBOPS.

Step 10: Web Pages Built

WEBOPS builds pages based on designs and GC copy.

WEBOPS adds links to master ticket and lets PROJECT MANAGER know they're ready for a QA review.

PROJECT MANAGER lets EDITORIAL, UX and DESIGN know pages are ready for review.

Step 11: Web Pages Reviewed

EDITORIAL and DESIGN review layout and content to ensure everything matches comps and GC. They provide feedback in appropriate webops tickets.

Once Complete, DESIGN & EDITORIAL inform the PROJECT MANAGER.

The PROJECT MANAGER sends links to STAKEHOLDER for review and updates master ticket with details.

STAKEHOLDER reviews links in master ticket and feedback (changes are limited to legal or factual errors) is provided in the following tools:

GatherContent for copy changes YouTrack Master Ticket for all other changes

Step 12: Final Updates

UX, EDITORIAL, DESIGN, and WEBOPS update per feedback and mark as "ready to launch" in appropriate webops ticket.

Step 13: Go No-go and Launching

GO NO-GO

PROJECT MANAGER schedules a "Go No-Go" meeting with all team members including STAKEHOLDER.
This is sometimes combined with the launch call if everything goes well.

Any changes during this meeting should be focused on:

Will Intel get sued if this goes live as is?

Will anyone get fired if this goes live as is?

Is something just plain wrong (typically a spec # or a link is going to the wrong place)?

PROJECT MANAGER, EDITORIAL, DESIGN, UX and WEBPOPS will:

• Check If feeds are showing the right content
• Check If all their changes have been implemented
• Ensure that all documentation and supporting content is pushed live

STAKEHOLDER will:

Check If links are going to the correct location

Anything else should be entered as a post-launch update ticket.

If everything is ready and you have enough time, the Go-No Go meeting can sometimes becomes the launch call. 

If the team needs more time, the PROJECT MANAGER schedules a launch call.

THE LAUNCH CALL

All team members attend and review content as it is pushed live.

UX, DESIGN, WEBOPS, PM will:

Check If links work (no 404 errors) Check If last minute changes are complete Check If layout, design, copy is correct. Check If all the redirects are set up

After launch, PROJECT MANAGER holds all tickets for localization while the site stabilizes (typically 2 weeks unless otherwise specified).

What is a Roofline Model?

A Roofline chart is a visual representation of application performance in relation to hardware limitations, including memory bandwidth and computational peaks. It was first proposed by researchers at the University of California at Berkeley in the 2009 paper “Roofline: An Insightful Visual Performance Model for Multicore Architectures.” In 2013, this model was improved by researchers at the Technical University of Lisbon in a paper called “Cache-Aware Roofline Model: Upgrading the Loft.” Traditionally, Roofline charts have been calculated and plotted manually, but Intel® Advisor now automatically builds Roofline plots.

The Roofline provides insight into:

• Where your performance bottlenecks are
• How much performance is left on the table because of them
• Which bottlenecks are possible to address, and which ones are worth addressing
• Why these bottlenecks are most likely occurring
• What your next steps should be

While the Roofline chart is not a conversion table that directly tells you exactly what changes need to be made in your code, it is an incredibly useful diagnosis tool. In the same way that a doctor uses a patient’s symptoms to determine what tests to run in order to diagnose the illness (and from there, what to prescribe), a developer can use the Roofline to guide them to what they should investigate next (and from there, what optimizations to make), using the rest of the tools provided in Intel® Advisor, ensuring that they can squeeze the maximum performance out of their code with minimal time and effort.

Understanding a Roofline Chart

Reading the Roofline

The Roofline is plotted with the X axis as Arithmetic Intensity (measured in FLOPs/Byte) and the Y axis as the performance in GFLOPs/Second, both in logarithmic scale. Before collecting data on your program, Intel® Advisor automatically runs some quick benchmarks to measure the hardware limitations of your machine, which it then plots as the lines on the chart, called roofs.

The horizontal lines represent the number of floating point computations of a given type your hardware can perform in a given span of time. The diagonal lines are representative of how many bytes of data a given memory subsystem can deliver per second.

Each dot is a loop or function in your program, with its position indicating its performance, which is affected by its optimization, and its Arithmetic Intensity. It is important to note that Intel® Advisor uses a cache-aware roofline model. In classic roofline, a kernel’s Arithmetic Intensity would change with problem size or cache usage optimization, because the byte count was based on DRAM traffic only. This is not so in cache-aware Roofline, where it is a fixed value tied to the algorithm itself – it only changes when the algorithm itself is altered, either by the programmer or occasionally by the compiler.

TIP: The Roofline chart in Intel® Advisor is highly customizable for maximum readability. You can show or hide specific roofs or alter the way loops/functions are displayed on the chart with the options menu accessible from the three horizontal lines in the upper right hand corner of the chart, as well as changing what data is displayed with Advisor's filter features.

Interpreting the Data

In general, the farther a dot is from the topmost roofs, the more room for improvement there is. In accordance with Amdahl’s Law, optimizing the loops that take the largest portion of the program’s total run time will lead to greater speedups than optimizing the loops that take a smaller portion of the run time.

The size and color of the dots in Intel® Advisor’s Roofline chart indicates how much of the total program time a loop or function takes. Small, green dots take up relatively little time, so are likely not worth optimizing. Large, red dots take up the most time, so the best candidates for optimization are the large, red dots with a large amount of space between them and the topmost attainable roofs.

In this example image, loops A, G, and (to a lesser extent) B are the best candidates for optimization, while loops E, C, D, and H are poor candidates.

The roofs above a dot represent the restrictions preventing it from achieving a higher performance, although the roofs below can contribute somewhat. Each roof represents the maximum performance achievable without taking advantage of a particular optimization, which is associated with the next roof up. For example, the Scalar Add Peak represents the maximum possible performance without taking advantage of vectorization, as indicated by the next roof up being the Vector Add Peak.

Of course, this means that some algorithms are incapable of breaking certain roofs. For instance, if Loop A in the example above cannot be vectorized due to dependencies, it cannot break the Scalar Add Peak.

TIP: If you can’t break a memory roof, try to rework your algorithm for higher arithmetic intensity. This will move you to the right and give you more room to increase performance before hitting the memory bandwidth roof. This would be the appropriate approach to optimizing loop F in the example, as well as loop G if its cache usage cannot be improved.

These are generalizations and simplifications, of course. In reality, the position of a loop on the chart is not always a direct indicator of what optimization to make, but it does provide a very reliable guide to what you should investigate next. A loop near (as B) or under (as A) the Scalar Add Peak indicates that you should check whether the loop vectorized – if so, you should investigate the vector efficiency; if not, you should investigate whether it can be vectorized at all. Likewise, a loop that is under a memory roof, such as G in the example, should have its memory access patterns investigated.

For a more in-depth explanation of how to interpret the Roofline results, watch the Roofline tutorial video.

Where can I get Intel® Advisor with Rooflines?

Roofline is a feature of Intel® Advisor beginning officially with version 2017 Update 2. It is also available as a preview feature in Intel® Advisor 2017 Update 1. Intel® Advisor is part of the Parallel Studio XE suite (Cluster Edition and Professional Edition only).

If you already own Intel® Advisor under a license that qualifies you for an update that includes the Roofline analysis, visit the Registration Center to download and install your update.

For more information on getting started with Intel® Advisor Roofline, see the Getting Started Guide.

Intel released update 2 to Intel® Cluster Checker 2017. This comprehensive diagnostic tool is distributed in Intel® Parallel Studio XE 2017 Cluster Edition Update 2 and supports Intel® Xeon™ processors, Intel® Xeon Phi™ processors, Intel® Omni-Path, and Intel® Enterprise Edition for Lustre* software. Here is a quick summary of what’s new:

Additional support for Intel® Xeon Phi™ Product Family x200 processors on configuration parameters that may have noticeable impact on system stability or performance:

• Identifying cluster and memory mode of Intel® Xeon Phi processors through backup-method with ‘numactl’ command, if hwloc-dump-data service is missing or not enabled.
• Taskset support for DGEMM/STREAM can be configured via xml configuration file.
• Check for tickless kernel boot options ‘nohz_full’, ‘rcu_nocbs’, ‘isolcpus
• Uniformity check
• Cross reference check to ensure ‘nohz_full’ is a subset of ‘rcu_nocbs’ and ‘isolcpus’ (if set)
• Desired kernel boot options check can be specified via xml configuration file.

 Additional support for Intel® Omni-Path Architecture:

• Two Intel® Omni-Path Host Fabric Interface (Intel® OP HFI) Adapters supported on one node.

Core improvements and bug fixes

The updated version contains several bug fixes when compared to the previous  Intel® Threading Building Blocks (Intel® TBB) 2017 Update 4 release. Information about new features of previous releases you can find under the following links.

•  What's new in Intel® Threading Building Blocks (Intel® TBB) 2017 Update 4

Added functionality:

• Added support for Microsoft* Visual Studio* 2017.
• Added graph/matmult example to demonstrate support for compute offload to Intel(R) Graphics Technology in the flow graph API.
• The "compiler" build option now allows to specify a full path to the compiler.

Changes affecting backward compatibility:

• Constructors for many classes, including graph nodes, concurrent containers, thread-local containers, etc., are declared explicit and cannot be used for implicit conversions anymore.

Bugs fixed:

• Added a workaround for bug 16657 in the GNU C Library (glibc) affecting the debug version of tbb::mutex.
• Fixed a crash in pool_identify() called for an object allocated in another thread.

You can download the latest Intel TBB version from http://threadingbuildingblocks.org and https://github.com/01org/tbb/releases. 

What to Download

By downloading a package from this page, you accept the End User License Agreement.

Installation has two parts:

• Intel® SDK for OpenCL™ Applications Package
• Driver and library(runtime) packages

The SDK includes components to develop applications.  Usually on a development machine the driver/runtime package is also installed for testing.  For deployment you can pick the package that best matches the target environment.

The illustration below shows some example install configurations. 

SDK Packages

Please note: A GPU/CPU driver package or CPU-only runtime package is required in addition to the SDK to execute applications

Standalone:

• Intel® SDK for OpenCL™ Applications 2016 R2 for Linux* (64-bit)  
• Intel® SDK for OpenCL™ Applications 2016 R2 for Windows* (64-bit) (assumes Windows* graphics driver installed)

Suite: (also includes driver and Intel® Media SDK)

• Intel® Media Server Studio 

Driver/Runtime Packages Available

GPU/CPU Driver Packages

• OpenCL™ 2.0 GPU Driver Package for Linux*(64-bit)
• (Also automatically shipped with Windows graphics drivers) 

CPU-only Runtime Packages  

• OpenCL™ Runtime for Intel® Core™ and Intel® Xeon® Processors

Intel® SDK for OpenCL™ Applications 2016 R3 for Linux (64-bit)

This is a standalone release for customers who do not need integration with the Intel® Media Server Studio. It provides components to develop OpenCL applications for Intel processors. 

Visit https://software.intel.com/en-us/intel-opencl to download the version for your platform. For details check out the Release Notes.

Intel® SDK for OpenCL™ Applications 2016 R3 for Windows* (64-bit)

This is a standalone release for customers who do not need integration with the Intel® Media Server Studio. The standard Windows graphics driver packages contains the driver and runtime library components necessary to run OpenCL applications. This package provides components for OpenCL development. 

Visit https://software.intel.com/en-us/intel-opencl to download the version for your platform. For details check out the Release Notes.

OpenCL™ 2.0 GPU/CPU driver package for Linux* (64-bit)

The Intel intel-opencl-r4.0 (SRB4) Linux driver package  provides access to the GPU and CPU components of these processors:

• Intel® 5th, 6th, or 7th generation Intel® Core™ processors
• Intel® Celeron® Processor J3000 Series with Intel® HD Graphics 500 (J3455, J3355), Intel® Pentium® Processor J4000 Series with Intel® HD Graphics 505 (J4205), Intel® Celeron® Processor N3000 Series with Intel® HD Graphics 500 (N3350, N3450), Intel® Pentium Processor N4000 Series with Intel® HD Graphics 505 (N4200)
• Intel® Xeon® processor v4 or v5 with Intel® Graphics Technology (if enabled by OEM in BIOS and motherboard)

Installation instructions

Intel has validated this package on CentOS 7.2 for the following 64-bit kernels.

• Linux 4.7 kernel patched for OpenCL*

Supported OpenCL devices:

• Intel® graphics (GPU)
• CPU

For detailed information please see the driver package Release Notes.

Previous Linux driver packages:

For Linux drivers covering earlier platforms such as 4th generation Intel Core processor please see the versions of Media Server Studio in the Driver Support Matrix.

OpenCL™ Driver for Iris™ graphics and Intel® HD Graphics for Windows* OS (64-bit and 32-bit)

The standard Intel graphics drivers for Windows* include components needed to run OpenCL* and Intel® Media SDK applications on processors with Intel® Iris™ Graphics or Intel® HD Graphics on Windows* OS.

You can use the Intel Driver Update Utility to automatically detect and update your drivers and software.  Using the latest available graphics driver for your processor is usually recommended.

Supported OpenCL devices:

• Intel graphics (GPU)
• CPU

For the full list of Intel® Architecture processors with OpenCL support on Intel Graphics under Windows*, refer to the Release Notes.

OpenCL™ Runtime for Intel® Core™ and Intel® Xeon® Processors

This runtime software package adds OpenCL CPU device support on systems with Intel Core and Intel Xeon processors.

Supported OpenCL devices:

• CPU

Latest release (16.1.1)

• OpenCL™ Runtime 16.1.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (64-bit & 32-bit)
• OpenCL™ Runtime 16.1.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (32-bit)
• OpenCL™ Runtime 16.1.1 for Intel® Core™ and Intel® Xeon® Processors for Red Hat* Linux* (64-bit)
• OpenCL™ Runtime 16.1.1 for Intel® Core™ and Intel® Xeon® Processors for SLES* Linux* (64-bit)
• OpenCL™ Runtime 16.1.1 for Intel® Core™ and Intel® Xeon® Processors for Ubuntu* (64-bit)

Previous Runtimes (16.1)

• OpenCL™ Runtime 16.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (64-bit & 32-bit)
• OpenCL™ Runtime 16.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (32-bit)
• OpenCL™ Runtime 16.1 for Intel® Core™ and Intel® Xeon® Processors for Red Hat* Linux* (64-bit)
• OpenCL™ Runtime 16.1 for Intel® Core™ and Intel® Xeon® Processors for SLES* Linux* (64-bit)
• OpenCL™ Runtime 16.1 for Intel® Core™ and Intel® Xeon® Processors for Ubuntu* (64-bit)

Previous Runtimes (15.1):

• OpenCL™ Runtime 15.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (64-bit & 32-bit)
• OpenCL™ Runtime 15.1 for Intel® Core™ and Intel® Xeon® Processors for Windows* (32-bit)
• OpenCL™ Runtime 15.1 for Intel® Core™ and Intel® Xeon® Processors for Red Hat* and SLES* Linux* (64-bit)

For the full list of supported Intel® architecture processors, refer to the OpenCL™ Runtime Release Notes.

 Deprecated Releases

Note: These releases are no longer maintained or supported by Intel

OpenCL™ Runtime 14.2 for Intel® CPU and Intel® Xeon Phi™ Coprocessors

This runtime software package adds OpenCL support to Intel Core and Xeon processors and Intel Xeon Phi coprocessors.

Supported OpenCL devices:

• Intel Xeon Phi coprocessor
• CPU

Available Runtimes

• OpenCL™ Runtime Driver for Intel® CPU and Intel® Xeon Phi™ Coprocessors for Linux* (64-bit)
• OpenCL™ Runtime Driver for Intel® CPU and Intel® Xeon Phi™ coprocessors for Windows* (64-bit)

For the full list of supported Intel architecture processors, refer to the OpenCL™ Runtime Release Notes.

test scheduled publishing

Solution providers achieve end-to-end integration, deliver actionable customer insights, and gain a competitive advantage with the Intel® IoT Platform and Google Cloud Platform*

A fully integrated edge-to-cloud IoT infrastructure solution can help to improve business insights that provide a true competitive advantage.

Executive Summary

Organizations that rely on the Internet of Things (IoT) for critical business processes are looking for ways to merge data silos, reduce security risks, and eliminate duplicate infrastructure. A fully integrated edge-to-cloud IoT infrastructure solution can help to improve business insights that provide a true competitive advantage. But implementing security-focused edge-to-cloud IoT solutions can be complex. Organizations with multiple IoT implementations need a planned approach to help ease that complexity. 

Intel and Google have worked together to deliver a standards-based approach to help IoT developers, OEMs, independent software vendors (ISVs), and system integrators (SI) develop seamless solutions. With a joint reference architecture built on the Intel® IoT Platform and the Google Cloud Platform* (GCP*), IoT providers can gain the following capabilities and benefits: 

• Seamless data ingestion. With a standards-based reference architecture, data is easier to collect and devices are easier to control. 
• End-to-end security. The architecture is designed to protect device hardware. 
• Easy device onboarding. New devices can be automatically provisioned to platforms, improving security. 
• Robust scalability. With Intel and Google technologies, organizations can scale rapidly on demand. 
• Better insights. GCP’s analytics infrastructure with Intel’s analytics-at-the-edge capabilities can provide better insights for faster decision making, quicker time-to-market, and the opportunity to provide new services and solutions. 

The Intel® IoT Platform and GCP joint reference architecture provides a more comprehensive approach for connecting the device layer to the network layer and into the cloud.

Figure 1. The joint Intel and Google reference architecture makes connecting the Internet of Things (IoT) from edge-to-cloud easier, with a focus on security at every layer.

Introduction

The Internet of Things (IoT) is speeding data collection from connected devices and sensors, resulting in an explosion of new devices and sensors that are generating massive volumes of data. This data can help organizations make smarter decisions and bring new products and services to market faster. Gartner Research estimates that by 2020, 25 billion enterprise-owned Internet-connected things across the globe stand to generate up to USD 2 trillion in economic benefit.1 This presents tremendous opportunities for IoT solution providers, but developing an edge-to-cloud solution can be complex. 

The technical challenges of IoT implementations often come from multiple IoT solutions dedicated to a variety of use cases within a single organization. These use cases can include monitoring chemical levels in manufacturing processes, occupancy-dependent lighting in offices, retail security cameras, or monitoring available parking. Multiple implementations also lead to a lack of interoperability between devices and equipment from different manufacturers. Successful IoT solutions require a deep understanding of infrastructure, security, integration, and interoperability from edge to cloud. Although IoT implementations can be complex, organizations and solution providers can eliminate much of the complexity and meet the growing IoT demand with integrated IoT solutions from Intel and Google. 

Solution Architecture

The Intel® IoT Platform and the Google Cloud Platform* (GCP*) each provide capabilities and benefits that help IoT developers, OEMs, independent software vendors (ISVs), and system integrators (SIs) develop industry-standard, seamless solutions.

Solution Overview and Benefits

Together, the Intel IoT and GCP joint reference architecture seamlessly transmits data from sensors, actuators, and other endpoint devices to the Google cloud. A clearly defined, standard reference architecture that details edge, network, and cloud components provides the following: 

• Seamless data ingestion and device control for improved interoperability.
• Robust security for end-to-end data and device protection. 
• Automated onboarding for simplified deployment of secure devices. 
• Robust scalability with cloud-based infrastructure.
• Customer insights through GCP’s analytics infrastructure.
• Data monetization through additional services and applications.

This joint reference architecture discusses:

• Intel IoT Platform. This illustrates the edge components, hardware security, and processors, as well as device provisioning, monitoring, and control. 
• Google Cloud Platform (GCP). This illustrates the cloud services, including data ingestion, flow, storage, and analytics.

The joint reference architecture is followed by an implementation overview, as well as a logistics and asset management use case example in Appendix A: Logistics and Asset Management Use Case. 

Intel® IoT Platform

The Intel IoT Platform (Figure 2) includes a family of Intel products. Intel’s IoT solution provider ecosystem delivers a foundation for easily connecting devices and delivering trusted data to the cloud. The Intel IoT Platform offers the following benefits:

• A broad array of devices. Intel’s ecosystem of original device manufacturers (ODMs) offers a wide range of devices and sensors built on Intel technology. 
• Security-focused solutions. Intel technology is designed for increased security at every layer, and includes zero-touch provisioning capabilities. 
• Enhanced registration and management. With Wind River Helix* Device Cloud, device management and updates are seamlessly controlled from a central point in the cloud. 

Google Cloud Platform* (GCP*)

GCP provides a secure, cost-effective, and high-performance infrastructure in the cloud hosted through Google’s globally distributed data centers (Figure 3). Managed services provide access this infrastructure for an overall solution. The benefits include: 

• Fully managed services. Google manages the setup and maintenance of the overall private infrastructure so customers can focus on building solutions. 
• Integrated development experience. GCP provides a wide range of services for an integrated, end-to-end developer experience.
• Full control of the environment. Developers have full control of their computing environment, from data ingestion to presentation, through APIs in multiple languages.
• Broad scale and reach. GCP offers outstanding scale and reach, resulting in a computing and data platform that is uniquely positioned to address the challenges of IoT.

Solution Architecture Details

The Intel IoT and GCP joint reference architecture (Figure 4) utilizes three primary types of components and solutions: Intel® Edge components, such as hardware security and processors; Intel device and security management, such as device provisioning, monitoring, and control; and GCP cloud services, such as data ingestion, dataflow, storage, and analytics. 

Intel® IoT Platform Components

Edge components

• Wind River Linux*. With built-in certifiable security capabilities and portability, Wind River* provides an IoT embedded Linux platform for hardware.
• Intel hardware-based security technologies. Capabilities such as secure boot, trusted execution environment (TEE), and Intel® Enhanced Privacy Identifier (Intel® EPID) secure the platform at the hardware level.
• Intel® Architecture Processors. Intel® Quark™ SE SoC and the Intel Atom®, Intel® Core™ processor, and Intel® Xeon® processor families provide high performance and scalability.

Device and security management

• Wind River Helix Device Cloud. Helix Device Cloud is an IoT portfolio of services and technologies that enable faster time to market; it provides device monitoring, control, software updates, registration, attestation, and secure deployment at scale. 
• Intel® Zero-Touch Device Onboarding. Using the privacy-preserving properties of Intel EPID—an IoT identity standard—and new onboarding protocols, owners can automatically register with their devices in GCP when powered on.

GCP Components

GCP components may vary depending on implementation and are grouped into five primary functions:

Data ingestion

• Cloud Pub/Sub*. Cloud Pub/Sub provides a fully managed, real-time messaging service that allows developers to send and receive messages between independent applications. 
• Cloud Stackdriver Monitoring*. Cloud Monitoring provides visibility into the performance, uptime, and overall health of cloud applications.
• Cloud Stackdriver Logging*. Cloud Logging allows developers to store, search, analyze, and monitor log data and events, as well as to send alerts.

Pipelines

• Cloud Dataflow*. Cloud Dataflow is a unified programming model the provides managed services for developing and executing a wide range of data processing patterns including extract, transform, load (ETL) and batch and continuous computation. Cloud Dataflow frees developers from operational tasks, such as resource management and performance optimization.

Storage

• Cloud Storage*. GCP provides an object store solution for excellent IoT performance and price.
• Cloud Datastore*. Cloud Datastore is a NoSQL database that is ideally suited for mobile and web endpoints. 
• Cloud Bigtable*. Cloud Bigtable is designed for workloads that require higher speed and lower latency, such as analytics.  

Analytics

• Cloud Dataflow*. Dataflow provides programming primitives, such as powerful windowing and correctness controls, that can be applied across both batch- and stream-based data sources.
• BigQuery*. BigQuery is a fully managed, petabyte-scale, low-cost enterprise data warehouse for analytics.
• Cloud Dataproc*. For Apache Spark* and Apache Hadoop*, Cloud Dataproc is designed for open source data tools for batch processing, querying, streaming, and machine learning.
• Cloud Datalab*. Cloud Datalab is an interactive tool for exploring, analyzing, and visualizing data with a single click.

Application and presentation

• App Engine*. App Engine is a platform-as-a-service (PaaS) solution used to develop applications without concern for the underlying infrastructure.
• Container Engine*. Container Engine is a managed Kubernetes* solution that provides industry-specific solutions, such as fleet management.
• Compute Engine*. Compute Engine is an infrastructure-as-a-service (IaaS) product that offers VMs on a variety of guest operating systems.

Implementation Overview

The process of connecting devices, integrating data, and managing software upgrades follows these steps (Figure 4): 

Onboarding Devices

1. During manufacturing, the silicon provider embeds Intel EPID credentials in a TEE of the processor. The ODM uses an open source toolkit from Intel to create a global unique identifier (GUID), assign a rendezvous URL for the device to “phone home” to get its new owner information, and generate an ownership proxy that is used to cryptographically verify ownership of the device by GCP.
2. Upon purchase, along with the purchase receipt, an ownership proxy for the device is generated. The owner imports the ownership proxy into GCP, which then signals to the Intel® cloud trust broker—a rendezvous service that directs a device to its new owner.
3. When the device is powered on the first time, it “phones home” to the Intel cloud trust broker, which redirects it to the IP address provided by its new designated GCP owner. 
4. The GCP trust broker and gateway verify the device through its Intel EPID signature and ownership proxy, and then register the device for management with the GCP and Wind River Helix Device Cloud.
5. The Wind River Helix Device Cloud distributes the device certificate provided by the GCP and configures the pub/sub topic subscriptions on the gateway.
6. The GCP IoT SDK on the gateway authenticates the GCP using the device certificate and establishes a secure data path to the GCP.

Collecting and Integrating Data

1. Business applications on the gateway acquire data from connected sensors through a number of supported protocols, such as Z-Wave*, ZigBee*, and Bluetooth®.
2. The GCP IoT SDK on the gateway transmits sensor data to the GCP through pub/sub messaging protocols.
3. Data messages are routed, processed, stored, and made available for enterprise integration. 

Managing Devices and Software Updates

1. Application software managers push updates to the Wind River Helix Device Cloud using APIs.
2. The Wind River Helix Device Cloud prepares signed RPM packages and pushes them securely to the gateway.
3. The management agent on the gateway of the Intel IoT Platform upgrades the software.

Summary

Intel and Google’s end-to-end joint reference architecture for secure IoT offers a robust, yet simplified solution that gives IoT developers the tools and services to create high-performance solutions. With secure, scalable interoperability, the Intel IoT and GCP joint reference architecture can provide the building blocks for any IoT application in any industry. 

The joint reference architecture is reusable, preconfigured, and prevalidated. It can connect devices with zero touch and deliver trusted data with interoperable hardware and software from the edge to the cloud. Each layer is designed with a focus on security and scalable hardware built on Intel technology is optimized for performance across workloads.

Find the solution that is right for your organization. Contact your Intel representative or visit intel.com/iot.

Learn More

You may also find the following resources useful:

• GCP Overview
• GCP Products 
• Intel IoT Gateway Technology
• Intel IoT Solutions 

Appendix A: Logistics and Asset Management Use Case

Having visibility to where shipments are at any given time is a significant pain point for supply chain businesses. Market research shows that approximately USD 60 billion worth of cargo is stolen during transit each year.² Additionally, roughly one third of the food produced in the world for human consumption every year gets lost or wasted.³ The ability to trace the journey of a package, such as high-value or perishable goods, in real-time can transform how companies manage, track, report, and secure products through logistics (shown in Figure A1). Table A1 illustrates an IoT solution using the Intel® IoT and Google Cloud Platform* (GCP*) joint reference architecture.

Figure A1. The Intel® IoT Platform and GCP* joint reference architecture provides visibility into the location of goods while in transit, helping transportation businesses reduce lost cargo.

Table A1. Technology Components for the IoT Shipment Visibility Use Case

1 gartner.com/smarterwithgartner/the-internet-of-things-and-the-enterprise

2 aic.gov.au/media_library/publications/tandi_pdf/tandi214.pdf

3 fao.org/save-food/resources/keyfindings/en

Software and workloads used in performance tests may have been optimized for performance only on Intel microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to intel.com/performance

Performance tests and ratings are measured using specific computer systems and/or components and reflect the approximate performance of Intel products as measured by those tests. Any difference in system hardware or software design or configuration may affect actual performance. Buyers should consult other sources of information to evaluate the performance of systems or components they are considering purchasing. For more information on performance tests and on the performance of Intel products, reference intel.com/performance/resources/benchmark_limitations or call (U.S.) 1-800-628-8686 or 1-916-356-3104.

All information provided here is subject to change without notice. Contact your Intel representative to obtain the latest Intel product specifications and roadmaps. Cost reduction scenarios described are intended as examples of how a given Intel- based product, in the specified circumstances and configurations, may affect future costs and provide cost savings. Circumstances will vary. Intel does not guarantee any costs or cost reduction. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software, or service activation. Performance varies depending on system configuration. No computer system can be absolutely secure. Check with your system manufacturer or retailer, or learn more at intel.com.

No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document.

THE INFORMATION PROVIDED IN THIS PAPER IS INTENDED TO BE GENERAL IN NATURE AND IS NOT SPECIFIC GUIDANCE. RECOMMENDATIONS (INCLUDING POTENTIAL COST SAVINGS) ARE BASED UPON INTEL’S EXPERIENCE AND ARE ESTIMATES ONLY. INTEL DOES NOT GUARANTEE OR WARRANT OTHERS WILL OBTAIN SIMILAR RESULTS.

INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL’S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.

Bluetooth is a trademark owned by its proprietor and used by Intel Corporation under license.

Copyright © 2017 Intel Corporation. All rights reserved. Intel, the Intel logo, Atom, Core, Quark, and Xeon are trademarks of Intel Corporation in the U.S. and/or other countries.

* Other names and brands may be claimed as the property of others.

Some customer may have a issue that the Visual Studio* (VS) text editor is blank and could not display content of code file after upgrading Intel® Parallel Studio XE (IPS). Here's the article talking about this problem and providing proper solution.

1. Issue Description

After upgrading Intel® IPS XE for windows which is integrated into Visual Studio*, the VS text editor is blank and could not display any content of code file including .c, .cpp, .h etc. But still could view content of xml and txt file.

2. To find Reason

There would be several reason for this problem, use below way to check the main reason:

a. Open any existed code file, click and drag the mouse in the window of editor to select a range of text, type ctrl-C to copy it, then paste into Notepad; If the content can be seen, follow solution a to fix.
b. Create a new code file, type into editor, save the file and use Notepad to see if there's some change. If so, please follow the solution b.
c. If above two ways are failed to test, please follow solution c.

3. Solution

a. It means the color of text is same as background color, please change front color to separate content and background.
b. It means the VS theme would be the main reason, try to change another theme.
c. It may caused by Visual Studio cache, you could try to delete all files under this folder (for instance, if you are using VS2015):
 C:\Users\<user name>\AppData\Local\Microsoft\VisualStudio\14.0\ComponentModelCache

Normally this kind of problems are not caused by Intel Parallel Studio XE. 

In recent years, machine learning and automation has taken over. Technology has adapted machines to the needs of a wide range of domains; even outperforming humans. Decisions made based only on life experience and intuition are no longer enough to grow a business. Instead, decisions based on past data are far more powerful and indicative of future trends. We as humans can extend our intelligence to train machines to imitate us, to do things as we ourselves would.

Furthermore, technology is advancing from software based on a certain set of rules predefined by humans, to platforms that can learn these rules by themselves, based on the data, and with much higher precision and speed than humans themselves.

In today’s fast-paced world, the competition is intense, even between the smallest of businesses. Retailers across the world cannot afford to ignore any insights about their products and services from current or potential customers. There was a time when we had to pay hefty amounts for this information, and the collection methods were often tedious. Long surveys, focus groups, and feedback forms were among the preferred ways for collecting customer insights. In spite of the pain and the cost there was a problem with these methodologies—the sample size was very small and therefore not always representative of the actual customer base. These methods were also intrusive and not regarded by the customers as a commitment toward offering better products and services. A loss-loss situation at best!

With the evolution of machine learning it becomes possible to use technology to capture customer sentiment without letting them know that they are being observed. We can now base our important business decisions on these rich customer insights and innovate continuously to offer solutions that align with customer needs.

Today we’ll take a close look at exactly how retailers are using machine learning technologies to maximize their business. To do so, we’ll talk about the application programming interface (API). If you have a technical background, chances are that you might be familiar with and using this important terminology.

If you are new to APIs, just follow along. We will walk you through the basics. We will explain an API, the types of APIs, and various APIs that can be used for building data science applications. Finally, we will discuss a simple yet powerful way in which retailers can use Twitter* data to gain value with just a few lines of code.

Contents:

1. Application Programming Interface
2. Types of API
3. Difference between API and Library
4. Popular APIs
5. Top Five APIs to Extract Data
6. Top Five APIs to Perform Cool Tasks
7. Using the Twitter API to Perform Opinion Mining
8. End Notes

1 Application Programming Interface 

The concept of API is straightforward. Just as we humans use a graphical user interface (GUI) or command-line interface (CLI) to interact with programs, computer programs use an application programming interface (API) to interact with other programs. The syntax, or the language in which this interaction takes place, is defined by the author of the API.

An API provides the framework for end users to incorporate programs written by other users into their own programs using simple steps, instead of rewriting the complete code themselves.

APIs make the life of a programmer simpler by allowing structured access to a program that was written and implemented by someone else. Generally, an API works as a function that provides the user the ability to pass certain inputs in a predefined format, and gives out the result in a predefined format, which is accessible to the person making the API call.

REST stands for representational state transfer. It is a lighter-weight alternative to simple object access protocol (SOAP) and web services description language (WSDL) XML-based API protocols. REST uses a client-server model, where the server is an HTTP server and the client sends an HTTP request (such as GET, POST, PUT, and DELETE), along with a URL and variable parameters that are URL-encoded. The URL describes the object to act upon and the server replies with a result code and valid JavaScript* object notation (JSON).

2 Types of API 

2.1 Web-based API

Web-based APIs act as an interface at the server or the browser end. Most modern web applications use these APIs. They allow the user to incorporate various functionalities into their web applications or websites using APIs authored by someone else.

For example, whenever you visit a company’s website and look for the Contact Us tab, you might find a map with tagged office locations on it. Most of the time all this functionality is implemented with the help of the Google Map* API and can be incorporated with just a few lines of code! This is how powerful an API can be.

Study the following code:

<!DOCTYPE html><html><head><link rel="stylesheet" href="/maps/documentation/javascript/demos/demos.css"></head><body><div id="map"></div><script>
      function initMap() {
        var sanjose = {lat: 37.33, lng: -121.88};
        var lasvegas = {lat: 36.16, lng: -115.13};

        var map = new google.maps.Map(document.getElementById('map'), {
          center: sanjose,
          scrollwheel: false,
          zoom: 5
        });

In this example, a developer is using the Google API for directions from San Jose to Las Vegas. This simple example shows that the developer does not need to learn how to draw a map, calculate distance, and so on, and is simply using the Google Maps API Directions. All the developer needs to input is longitude and latitude of source and destination. In simple terms, there is a contract between the developer of this code and Google API—you give me longitude and latitude and I will give back a map with directions.

2.2 Operating system API

Creating an application for a Windows* or Mac* OS would be a very complex task. But thanks to the operating system APIs, a wide range of functionalities are provided that make the creation of Windows or Mac applications easier. The following diagram shows the Linux* API, which is composed of the system call interface of the Linux kernel.

2.3 Database API

Today, requests and response to almost all databases are handled using APIs. Instead of writing a complete query every time an update or retrieval is needed, a simple command to update the database or to retrieve the data from it can be used.

The request is passed to the database in this predefined format and the database server provides the response to the request in a format defined by the author of that API. These APIs make it rather easy to communicate with databases and ensure standardization and speed.

2.4 Hardware API

Hardware APIs act as a crucial connection between the hardware and software worlds. From detecting the low fuel status in your car to detecting the tire pressure of a jumbo jet, APIs are used extensively in a wide range of domains.

The hardware generally passes along the information it recorded in a format that can be captured and processed with the help of software.

Look at the following code using the Python* OBD library. Python OBD is a library for handling data from a car's on-board diagnostics port (OBD-II). It can stream real-time sensor data and perform diagnostics (such as reading check-engine codes). This library is designed to work with standard ELM327 OBD-II adapters.

3 Difference between API and Library 

As this article progresses, you may feel that APIs and libraries are the same things, when in fact they are not. So, it becomes crucial to explain the difference at this stage.

An API is simply an interface that can establish a communication between two applications. Certain rules and protocols govern this communication. The expected behavior of the API is predefined by its author, and anyone who wants to use it must follow these rules and protocols.

A library is much bigger than that. In fact, a library also has an API, which signifies the region in the library that is accessible to the outside user for communication or interaction. If the input is as specified, the output generated will follow the same format as specified.

4 Popular APIs 

Now, let’s look at some of the most popular APIs in the data science domain. We’ll break these APIs into two categories:

1. APIs to extract data: Those APIs which give us access to datasets on which we can build cool data science projects.
2. APIs to perform cool analysis: APIs to which we pass the data to perform very complex operations and return the results of that operation.

5 Top Five APIs to Extract Data 

Gone are the days when we need to work in a top firm at high positions to get data. Today, even the most valued datasets are available online for free. All credit goes to the developers who have worked hard in making the access to these datasets easy by creating APIs for them.

A list of popular category 1 APIs that extract data:

• Facebook* API.
• Twitter API.
• YouTube* API.
• GitHub* API.
• Instagram* API.

Figure 1.http://www.programmableweb.com/api/instagram.

6 Top Five APIs to Perform Cool Tasks 

A list of popular category 2 APIs that perform analysis:

• IBM Watson* API
• BigML* API
• Microsoft Azure* Cognitive Services API
• Google Cloud* Prediction API
• Prediction.io* API

Figure 2. Google Prediction API.

7 Using the Twitter API for Opinion Mining 

Now, we will take you through the implementation of opinion mining on data retrieved using the Twitter API. It will give you the power to know the opinions of people on any topic that you desire, across the globe, while sitting in your comfortable chair. Sounds interesting, right?

Let’s get started.

library('twitteR')
library('ROAuth')
        #set up twitter authentication
    consumerKey="XXX"
    consumerSecret="XXX"
    accessURL="XXX"
    authURL="XXX"
    reqURL="XXX"
    Cred <- OAuthFactory$new(consumerKey=consumerKey,
                             consumerSecret=consumerSecret,
                             requestURL=reqURL,
                             accessURL=accessURL,
                             authURL=authURL)

setup_twitter_oauth(consumerKey,consumerSecret,access_token=NULL,access_secret=NULL)

#Select search term
        searchterm<-“#iphone7”

#Select no. of tweets to invoke
        num<-1500

#Invoke tweets
        list <- searchTwitter(searchterm, n= num, lang="en", since=NULL, until=NULL, retryOnRateLimit=10)

        library("wordcloud")
        library("tm")

        l <- sapply(list, function(x) x$getText())

        l <- iconv(l, "latin1", "ASCII//TRANSLIT")


        l <- iconv(l, to='ASCII//TRANSLIT')

       #create corpus
        lc <- Corpus(VectorSource(l))

        #clean up

#Convert every word to lower
        lc <- tm_map(lc, content_transformer(tolower))

#Remove punctuation
        lc <- tm_map(lc, removePunctuation)

#Remove stop words
        lc <- tm_map(lc, function(x)removeWords(x,stopwords()))

        library(RColorBrewer)
        pal2 <- brewer.pal(8,"Dark2")

#create word cloud
        wordcloud(lc,min.freq=num/200,max.words=500, random.order=T, colors=pal2)

Output:

The output for the above code is generated in the form of a word cloud. This word cloud contains all the terms that are being frequently used in the tweets with your search term. Note that the stop words have been removed from the tweets. Each word that you see in the word cloud contains tons of importance. You can know the views of people across the globe on any topic that you wish.

Retailers use this technology extensively to know what their customers are thinking. It gives them an edge over their competitors and they can use the power of insights to grow their business.

8 End Notes 

APIs provide a lot of convenience to programmers. Instead of having to code every functionality that they want to implement in their applications, they can simply use APIs authored by someone else to do it for them.

A lot of social media giants are making their data publicly available. This opens a very exiting dimension which can provide deep business insights and market trends. It especially opens an opportunity in the field of natural language processing (NLP) in which retailers are getting to know their customers’ opinions and sentiments in a much-advanced manner.

References for images and research:

https://developers.google.com/maps/

https://en.wikipedia.org/wiki/Linux_kernel_interfaces#Linux_API

http://python-obd.readthedocs.io/en/latest/#welcome

https://www.r-project.org/about.html

http://www.programmableweb.com/news/apps-get-smarter-google-prediction-api/2010/09/22

http://triangles.net/instagram-api-and-what-it-means-for-brands/

How does a company forecast how much it can spend on future advertising in order to increase sales? Similarly, how much should a company spend in future training programs in order to improve productivities?

In both cases, those companies rely on the historical relationship between variables like, in the former case, the relationship between the sale value and advertising spending to predict the future outcome.

This is a typical regression problem that can be best solved using machine learning¹.

This article describes a common type of regression analysis called linear regression² and how the Intel® Data Analytics Acceleration Library (Intel® DAAL)³ helps optimize this algorithm when running it on systems equipped with Intel® Xeon® processors.

What is Linear Regression?

Linear regression (LR) is the most basic type of regression that is used for predictive analysis. LR shows a linear relationship between variables and how one variable can be affected by one or more variables. The variable which is impacted by the others is called a dependent, response, or outcome variable and the others are called independent, explanatory, or predictor variables.

In order to use LR analysis, we need to examine whether LR is suitable for this set of data. To do that we need to observe how the data is distributed. Let's look at the following two graphs:

 
Figure 1:A straight line can be fitted through the data points.

       
Figure 2:A straight line cannot be fitted through these data points.

In figure 1, we can fit a straight line through the data points; however, there is no way to do that with the data points in figure 2. Only the curved (non-linear) line can be fitted through the data points in figure 2. Therefore, linear regression analysis can be done on the dataset in figure 1, but not on that in figure 2.

Depending on the number of independent variables, LR is divided into two types: simple linear regression (SLR) and multiple linear regression (MLR).

LR is called SLR when there exists only one independent variable, whereas MLR has more than one independent variable.

The simplest form of the equation with one dependent and one independent variable is defined as:

y = Ax + B      (1)

Where:

y: Dependent variable

x: Independent variable

A: Regression coefficient or slope of the line

B: Constant

The question is how to find the best-fit line so that the difference between the observed and predicted values of the dependent variable (y) is minimum. In other words, find A and B in equation (1) such that |y_observed– y_predicted| is minimum.

The task to find the best-fit line can be done using the least squares method⁴. It calculates the best-fit line by minimizing the sum of the squares of the vertical differences (difference between the observed and predicted values of y) from each data point to the line. The vertical difference can also be called residual.

Figure 3:Linear regression graph.

From figure 3 the green dots represent the actual data points. The black dots show the vertical (not perpendicular) projection of the data points onto the regression line (red line). The black dots are also called the predicted data. The vertical difference between the actual data and the predicted data is called residual.

Applications of Linear Regression

Some of the applications that can make good use of linear regression:

• Forecasting future sales.
• Analyzing the marketing effectiveness and pricing on sales of a product.
• Assessing risk in financial services or insurance domains.
• Studying engine performance from test data in automobiles.

Advantages and Disadvantages of Linear Regression

Some of the pros and cons of LR:

• Advantages
  • The result is optimum when the relationship between the independent and the dependent variables are almost linear.
• Disadvantages
  • LR is very sensitive to outliers.
  • It is inappropriately used to model non-linear relationships.
  • Linear regression is limited to predicting numeric output.

Intel® Data Analytics Acceleration Library

Intel DAAL is a library consisting of many basic building blocks that are optimized for data analytics and machine learning. These basic building blocks are highly optimized for the latest features of the latest Intel® processors. LR is one of the predictive algorithms that Intel DAAL provides. In this article, we use the Python* API of Intel DAAL to build a basic LR predictor. To install DAAL, follow the instructions in How to install the Python Version of Intel DAAL in Linux*⁵.

Using Linear Regression Algorithm in Intel Data Analytics Acceleration Library

This section shows how to invoke the linear regression method in Python⁶ using Intel DAAL.

The reference section provides a link⁷ to the free datasets that can be used to test the application.

Do the following steps to invoke the algorithm from Intel DAAL:

1. Import the necessary packages using the commands from and import.
   1. Import Numpy by issuing the command:

      import numpy as np

   2. Import the Intel DAAL numeric table by issuing the following command:

      from daal.data_management import HomogenNumericTable

   3. Import necessary functions to numeric tables to store data:

      from daal.data_management import ( DataSourceIface, FileDataSource, HomogenNumericTable, MergeNumericTable, NumericTableIface)

   4. Import the LR algorithm using the following commands:

      from daal.algorithms.linear_regression import training, prediction

2. Initialize the file data source if the data input is from the .csv file:

   trainDataSet = FileDataSource(
           trainDatasetFileName, DataSourceIface.notAllocateNumericTable,
           DataSourceIface.doDictionaryFromContext
       )

3. Create numeric tables for training data and dependent variables:

   trainInput = HomogenNumericTable(nIndependentVar, 0, NumericTableIface.notAllocate)
   
   trainDependentVariables = HomogenNumericTable(nDependentVariables, 0, NumericTableIface.notAllocate)
   
   mergedData = MergedNumericTable(trainData, trainDependentVariables)
   

4. Loading input data:

   trainDataSet.loadDataBlock(mergedData)

5. Create a function to train the model.
   1. First create an algorithm object to train the model using the following command:

      algorithm = training.Batch_Float64NormEqDense()

      Note: This algorithm uses the normal equation to solve the linear least squares problem. DAAL also supports QR decomposition/factorization.
   2. Pass the training dataset and dependent variables to the algorithm using the following commands:

      algorithm.input.set(training.data, trainInput)
      algorithm.input.set(training.dependentVariables, trainDependentVariables)

      where
                algorithm: The algorithm object as defined in step a above. train
                Input: Training data. trainDependent
                Variables: Training dependent variables
   3. Train the model using the following command:

      trainResult = algorithm.compute()

      where
                algorithm: The algorithm object as defined in step a above.
6. Create a function to test the model.
   1. Similar to steps 2, 3, and 4 above, we need to create the test dataset for testing:
      i.

      testDataSet = FileDataSource(
      testDatasetFileName, DataSourceIface.doAllocateNumericTable,
      DataSourceIface.doDictionaryFromContext
       )

      ii.

      testInput = HomogenNumericTable(nIndependentVar, 0, NumericTableIface.notAllocate)
      testTruthValues = HomogenNumericTable(nDependentVariables, 0, NumericTableIface.notAllocate)
      mergedData = MergedNumericTable(testDataSet, testTruthValues)

      iii.

      testDataSet.loadDataBlock(mergedData)

   2. Create an algorithm object to test/predict the model using the following command:

      algorithm = prediction.Batch()

   3. Pass the testing data and the train model to the model using the following commands:

      algorithm.input.setTable(prediction.data, testInput)
      algorithm.input.setModel(prediction.model, trainResult.get(training.model))

      where
                algorithm: The algorithm object as defined in step a above.
                testInput: Testing data.
   4. Test/predict the model using the following command:

      Prediction = algorithm.compute()

Conclusion

Linear regression is a very common predictive algorithm. Intel DAAL optimized the linear regression algorithm. By using Intel DAAL, developers can take advantage of new features in future generations of Intel Xeon processors without having to modify their applications. They only need to link their applications to the latest version of Intel DAAL.

References

1. Wikipedia – machine learning
2. Linear regression
3. Introduction to Intel Data Analytics Acceleration Library
4. Least Squares Method
5. How to install the Python Version of Intel DAAL in Linux
6. Python website
7. List of common datasets

Download PDF 957KB

This article describes the Data Plane Development Kit (DPDK) Packet Framework, which allows rapid prototyping of real-world packet processing workloads on multicore Intel® CPUs with great flexibility and performance. This framework is based on three DPDK libraries: librte_port, librte_table, and librte_pipeline.

The DPDK Packet Framework library was first introduced in DPDK v2.1 along with a reference application known as ip_pipeline. In subsequent releases, various features were added to the ip_pipeline application to transform it into a toolbox for developing complex packet processing workloads such as edge routers. The information here applies to DPDK versions 16.04 and above.

Packet Framework Library 

The DPDK Packet Framework employs a suite of DPDK libraries (librte_port, librte_table, and librte_pipeline) to define a standard methodology for building complex packet processing applications.  It provides reusable and extensible templates for building various functional application blocks as pipeline modules.

Each pipeline module is constructed of three primitive entities: input ports, output ports, and tables. An example is shown in Figure 1. 

Figure 1. The DPDK Packet Framework.

Input ports can only be connected to one table; however, that one table can be connected to other tables and also to output ports.

Various actions can be performed on packets as they traverse the pipeline blocks. For example, packet header integrity and verification can be performed at an input port. In addition, table actions can be applied, based on the result of a look-up operation, to forward packets to another table, to send packets to an output port, or to simply drop the packets.

The IP Pipeline Application

The IP Pipeline application is designed to demonstrate the use of the DPDK Packet Framework tool suite. This sample application, in the DPDK codebase, provides the following data plane functions implemented using the above pipeline model.

• Pass-through Pipeline: This pipeline offers a cable-like connectivity between its input ports and output ports and is typically used to adapt a set of ports from one type to another. This pipeline is basically a placeholder for implementing various functions such as load balancing or computing metadata required at subsequent packet processing pipeline stages.
• Flow Classification Pipeline: This pipeline connects all the input ports to a table that classifies the packets based on a hash lookup and sends them to the matching ports. Different types of packets such as QinQ, IPv4, and IPv6 can be classified using this pipeline.
• Routing Pipeline: This pipeline employs a longest prefix match (LPM) table to route the incoming packets. If fast path address resolution protocol (ARP) (implemented using a hash table) is enabled in the pipeline, the output port is determined on the basis of LPM and ARP table lookups. Depending on the next hop protocol, the packet can also be encapsulated with a QinQ header or multiprotocol label switching (MPLS) labels.
• Firewall Pipeline: This pipeline uses an access control list (ACL) table to implement firewall policies on the incoming traffic.
• Flow Action: This pipeline performs two operations:
  • Ingress traffic metering/marking by assigning colors to the packets as a measure of how much the user/flow currently exceeds its pre-allocated bandwidth.
  • Traffic policing by enforcing predefined actions on the input packet based on the assigned color.

Theses pipelines can be seen as prefabricated blocks that can be instantiated and interconnected by means of a configuration file to create complete applications.

The application configuration file defines the application structure that includes pipeline instances, device queues, and software queues. Different applications are created by using different configuration files; therefore, the ip_pipeline sample application can be seen as an application generator. The configuration of each pipeline can be updated at run time through the application's command-line interface (CLI).

Running the IP Pipeline Sample Application

The following steps are required to run the DPDK Packet Framework-based ip_pipeline sample application.

• Go to the ip_pipeline application directory:

$ cd /home/dpdk/examples/ip_pipeline/

• Export the following environmental variables and set the required target:

$ export RTE_SDK = /home/dpdk

$ export RTE_TARGET = x86_64-native-linuxapp-gcc

• Build the ip_pipeline application:

$ make

• To run the ip_pipeline sample use a command line like the following:

ip_pipeline [-f CONFIG_FILE] [-s SCRIPT_FILE] -p PORT_MASK [-l LOG_LEVEL]

The application command-line arguments are:

• -f CONFIG_FILE: (optional, default: ./config/ip_pipeline.cfg). This is the path to the configuration file to be loaded by the application.
• -s SCRIPT_FILE: (optional). This is the path to the CLI script file to be run by the master pipeline at application startup. No CLI script file will be run at startup if this argument is not present.
• -p PORT_MASK: (required). The hexadecimal mask of the NIC port IDs to be used by the application.
• -l LOG_LEVEL: (optional). The log level to determine which application messages are to be printed to standard output. Available log levels are: 0 (None), 1 (High priority), and 2 (Low priority).

Example: L3 Forwarding

The usage of the pipeline blocks is demonstrated by means of several sample applications and configuration files that are provided with the ip_pipeline application. 

Figure 2 shows one such configuration file for a Layer 3 (L3) forwarding application.

Figure 2: L3 Forwarding Application Configuration file (l3fwd.cfg).

In the file, the [PIPELINE] section defines various parameters of the pipeline. In this particular case it sets up simple routing.

The "core" entries thread id (tuple of socket ID, physical CPU ID, and hyper-thread ID) determines the CPU core to run the pipeline.

The "pktq_in" and "pktq_out" parameters define the packet transfer interface; in this case, to receive and send packets.

The "encap" parameter can be set to "ethernet", "qinq", or "mpls" to encapsulate all outgoing packets with the appropriate header.

The last parameter, "ip_hdr_offset", is used to set the offset bytes needed to offset to the start of the ip-header in the DPDK packet structure (mbuf).

Figure 3 shows the ip_pipeline script file with routing rules added to the LPM table of the routing pipeline. Entries in this file are related to the case where the "encap" field is set to "ethernet" and the fast path ARP is disabled. 

           
Figure 3: L3 Forwarding Application Rules file (l3fwd.sh).

The rules in the script file have the following format:

p <pipeline_id> route add <ip_addr> <depth> port <port_id> ether <next hop mac_addr>

Use the following command to run the L3 forwarding application:

$./build/ip_pipeline -f l3fwd.cfg -p 0xf -s l3fwd.sh

Visual Representation

Sometimes applications can have complex topologies in terms of interconnections between different functional blocks. In these cases it can becomes difficult to understand the application packet processing flow by simply looking at its configuration file. In order to help with this a Python* program called diagram‑generator.py has been provided to create visual diagrams of the application topology from the configuration file.

Once the application configuration file is ready, run the command shown below to generate the diagram.

$./diagram-generator.py -f <configuration file>

Figure 4 shows a section of a generated topology for a complex ip_pipeline application.

Figure 4: Section of a generated topology for a complex ip_pipeline application.

Conclusion

This article describes the DPDK packet framework library, which enables rapid prototyping of real-world packet processing applications on multicore CPU systems, and presents the architecture of the standard pipeline model built using this library.

The various network functions (pipelines) constructed using this standard pipeline model are available as part of the DPDK ip_pipeline sample application. Although some of these are simple examples, a straightforward configuration file can build realistic applications. This article presents the commands to visualize and run the applications.

Additional Information

Check out the DPDK Programmer's Guide for a full description of the DPDK Packet Framework design.

Two sample applications are provided for Packet Framework: app/test-pipeline and examples/ip_pipeline. Refer to the DPDK Sample Apps Guide for a detailed description of how these sample applications work.

About the Author

Jasvinder Singh is a network software engineer with Intel. His work is primarily focused on development of data plane functions and libraries for DPDK. His contributions include packet framework enhancements and the ip_pipeline sample application.

• How can I get started with Intel XDK?
• How do I convert my web app or web site into a mobile app?
• Can I use an external editor for development in Intel® XDK?
• How do I get code refactoring capability in Brackets* (the Intel XDK code editor)?
• Why doesn’t my app show up in Google* play for tablets?
• What are the global-settings.xdk and xdk.log files?
• When do I use the intelxdk.js, xhr.js and cordova.js libraries?
• How do I get my Android (and Crosswalk) keystore file?
• How do I rename my project that is a duplicate of an existing project?
• How do I recover when the Intel XDK hangs or won't start?
• Is Intel XDK an open source project? How can I contribute to the Intel XDK community?
• How do I configure Intel XDK to use 9 patch png for Android* apps splash screen?
• How do I stop AVG from popping up the "General Behavioral Detection" window when Intel XDK is launched?
• What do I specify for "App ID" in Intel XDK under Build Settings?
• Is it possible to modify the Android Manifest or iOS plist file with the Intel XDK?
• How can I share my Intel XDK app build?
• Why does my iOS build fail when I am able to test it successfully on a device and the emulator?
• How do I add multiple domains in Domain Access?
• How do I build more than one app using the same Apple developer account?
• How do I include search and spotlight icons as part of my app?
• Does Intel XDK support Modbus TCP communication?
• How do I sign an Android* app using an existing keystore?
• How do I build separately for different Android* versions?
• How do I display the 'Build App Now' button if my display language is not English?
• How do I update my Intel XDK version?
• How do I import my existing HTML5 app into the Intel XDK?
• I am unable to login to App Preview with my Intel XDK password.
• How do I completely uninstall the Intel XDK from my system?
• Is there a tool that can help me highlight syntax issues in Intel XDK?
• How do I delete built apps and test apps from the Intel XDK build servers?
• I need help with the App Security API plugin; where do I find it?
• When I install my app or use the Debug tab Avast antivirus flags a possible virus, why?
• How do I add a Brackets extension to the editor that is part of the Intel XDK?
• Why does my app or game require so many permissions on Android when built with the Intel XDK?
• How do I make a copy of an existing Intel XDK project?
• My project does not include a www folder. How do I fix it so it includes a www or source directory?
• Can I install more than one copy of the Intel XDK onto my development system?
• On Apple OS X* and Linux* systems, does the Intel XDK need the OpenSSL* library installed?
• I have a web application that I would like to distribute in app stores without major modifications. Is this possible using the Intel XDK?
• What is the best training approach to using the Intel XDK for a newbie?
• What is the best platform to start building an app with the Intel XDK? And what are the important differences between the Android, iOS and other mobile platforms?
• Is my password encrypted and why is it limited to fifteen characters?
• Why does the Intel XDK take a long time to start on Linux or Mac?
• How do I generate a P12 file on a Windows machine?
• How do I change the default dir for creating new projects in the Intel XDK?
• Where I can find recent and upcoming webinars list?
• How can I change the email address associated with my Intel XDK login?
• What network addresses must I enable in my firewall to insure the Intel XDK will work on my restricted network?
• I cannot create a login for the Intel XDK, how do I create a userid and password to use the Intel XDK?
• Installing the Intel XDK on Windows fails with a "Package signature verification failed." message.
• Troubles installing the Intel XDK on a Linux or Ubuntu system, which option should I choose?
• Inactive account/ login issue/ problem updating an APK in store, How do I request account transfer?
• Connection Problems? -- Intel XDK SSL certificates update
• How do I resolve build failure: "libpng error: Not a PNG file"?  
• Why do I get a "Parse Error" when I try to install my built APK on my Android device?
• My converted legacy keystore does not work. Google Play is rejecting my updated app.
• How can I have others beta test my app using Intel App Preview?
• I'm having trouble making Google Maps work with my Intel XDK app. What can I do?
• How do I fix "Cannot find the Intel XDK. Make sure your device and intel XDK are on the same wireless network." error messages?
• Google Services needs my SHA1 fingerprint. Where do I get my app's SHA fingerprint?
• Why am I unable to test or build or connect to the old build server with Intel XDK version 2893?
• How do I run the Intel XDK on Fedora Linux?
• The Intel XDK generates a path error for my launch icons and splash screen files.
• Upgrading to the latest version of the Intel XDK results in a build error with existing projects.

How can I get started with Intel XDK?

There are plenty of videos and articles that you can go through here to get started. You could also start with some of our demo apps. It may also help to read Five Useful Tips on Getting Started Building Cordova Mobile Apps with the Intel XDK, which will help you understand some of the differences between developing for a traditional server-based environment and developing for the Intel XDK hybrid Cordova app environment.

Having prior understanding of how to program using HTML, CSS and JavaScript* is crucial to using the Intel XDK. The Intel XDK is primarily a tool for visualizing, debugging and building an app package for distribution.

You can do the following to access our demo apps:

• Select Project tab
• Select "Start a New Project"
• Select "Samples and Demos"
• Create a new project from a demo

If you have specific questions following that, please post it to our forums.

How do I convert my web app or web site into a mobile app?

The Intel XDK creates Cordova mobile apps (aka PhoneGap apps). Cordova web apps are driven by HTML5 code (HTML, CSS and JavaScript). There is no web server in the mobile device to "serve" the HTML pages in your Cordova web app, the main program resources required by your Cordova web app are file-based, meaning all of your web app resources are located within the mobile app package and reside on the mobile device. Your app may also require resources from a server. In that case, you will need to connect with that server using AJAX or similar techniques, usually via a collection of RESTful APIs provided by that server. However, your app is not integrated into that server, the two entities are independent and separate.

Many web developers believe they should be able to include PHP or Java code or other "server-based" code as an integral part of their Cordova app, just as they do in a "dynamic web app." This technique does not work in a Cordova web app, because your app does not reside on a server, there is no "backend"; your Cordova web app is a "front-end" HTML5 web app that runs independent of any servers. See the following articles for more information on how to move from writing "multi-page dynamic web apps" to "single-page Cordova web apps":

• SPA is Your Friend
• Converting a dynamic web site to a PhoneGap application

Can I use an external editor for development in Intel® XDK?

Yes, you can open your files and edit them in your favorite editor. However, note that you must use Brackets* to use the "Live Layout Editing" feature. Also, if you are using App Designer (the UI layout tool in Intel XDK) it will make many automatic changes to your index.html file, so it is best not to edit that file externally at the same time you have App Designer open.

Some popular editors among our users include:

• Sublime Text* (Refer to this article for information on the Intel XDK plugin for Sublime Text*)
• Notepad++* for a lighweight editor
• Jetbrains* editors (Webstorm*)
• Vim* the editor

How do I get code refactoring capability in Brackets* (the Intel XDK code editor)?

...to be written...

Why doesn’t my app show up in Google* play for tablets?

...to be written...

Where are the global-settings.xdk and xdk.log files?

global-settings.xdk contains information about all your projects in the Intel XDK, along with many of the settings related to panels under each tab (Emulate, Debug etc). For example, you can set the emulator to auto-refresh or no-auto-refresh. Modify this file at your own risk and always keep a backup of the original!

The xdk.log file contains logged data generated by the Intel XDK while it is running. Sometimes technical support will ask for a copy of this file in order to get additional information to engineering regarding problems you may be having with the Intel XDK. 

Both files are located in the same directory on your development system. Unfortunately, the precise location of these files varies with the specific version of the Intel XDK. You can find the global-settings.xdk and the xdk.log using the following command-line searches:

• From a Windows cmd.exe session:
  > cd /
> dir /s global-settings.xdk
   
• From a Mac and Linux bash or terminal session:
  $ sudo find / -name global-settings.xdk

When do I use the intelxdk.js, xhr.js and cordova.js libraries?

The intelxdk.js and xhr.js libraries were only required for use with the Intel XDK legacy build tiles (which have been retired). The cordova.js library is needed for all Cordova builds. When building with the Cordova tiles, any references to intelxdk.js and xhr.js libraries in your index.html file are ignored.

How do I get my Android (and Crosswalk) keystore file?

New with release 3088 of the Intel XDK, you may now download your build certificates (aka keystore) using the new certificate manager that is built into the Intel XDK. Please read the initial paragraphs of Managing Certificates for your Intel XDK Account and the section titled "Convert a Legacy Android Certificate" in that document, for details regarding how to do this.

It may also help to review this short, quick overview video (there is no audio) that shows how you convert your existing "legacy" certificates to the "new" format that allows you to directly manage your certificates using the certificate management tool that is built into the Intel XDK. This conversion process is done only once.

If the above fails, please send an email to html5tools@intel.com requesting help. It is important that you send that email from the email address associated with your Intel XDK account.

How do I rename my project that is a duplicate of an existing project?

See this FAQ: How do I make a copy of an existing Intel XDK project?

How do I recover when the Intel XDK hangs or won't start?

• If you are running Intel XDK on Windows* you must use Windows* 7 or higher. The Intel XDK will not run reliably on earlier versions.

• Delete the "project-name.xdk" file from the project directory that Intel XDK is trying to open when it starts (it will try to open the project that was open during your last session), then try starting Intel XDK. You will have to "import" your project into Intel XDK again. Importing merely creates the "project-name.xdk" file in your project directory and adds that project to the "global-settings.xdk" file.

• Rename the project directory Intel XDK is trying to open when it starts. Create a new project based on one of the demo apps. Test Intel XDK using that demo app. If everything works, restart Intel XDK and try it again. If it still works, rename your problem project folder back to its original name and open Intel XDK again (it should now open the sample project you previously opened). You may have to re-select your problem project (Intel XDK should have forgotten that project during the previous session).

• Clear Intel XDK's program cache directories and files.
  
  On a Windows machine this can be done using the following on a standard command prompt (administrator is not required):
  
  > cd %AppData%\..\Local\XDK
> del *.* /s/q
  
  To locate the "XDK cache" directory on [OS X*] and [Linux*] systems, do the following:
  
  $ sudo find / -name global-settings.xdk
$ cd <dir found above>
$ sudo rm -rf *
  
  You might want to save a copy of the "global-settings.xdk" file before you delete that cache directory and copy it back before you restart Intel XDK. Doing so will save you the effort of rebuilding your list of projects. Please refer to this question for information on how to locate the global-settings.xdk file.

• If you save the "global-settings.xdk" file and restored it in the step above and you're still having hang troubles, try deleting the directories and files above, along with the "global-settings.xdk" file and try it again.

• Do not store your project directories on a network share (the Intel XDK has issues with network shares that have not been resolved). This includes folders shared between a Virtual machine (VM) guest and its host machine (for example, if you are running Windows* in a VM running on a Mac* host). 

• Some people have issues using the Intel XDK behind a corporate network proxy or firewall. To check for this issue, try running the Intel XDK from your home network where, presumably, you have a simple NAT router and no proxy or firewall. If things work correctly there then your corporate firewall or proxy may be the source of the problem.

• Issues with Intel XDK account logins can also cause Intel XDK to hang. To confirm that your login is working correctly, go to the Intel login page and confirm that you can login with your Intel XDK account username and password.

• If you are experiencing login issues, please send an email to html5tools@intel.com from the email address registered to your login account, describing the nature of your account problem and any other details you believe may be relevant.

If you can reliably reproduce the problem, please post a copy of the "xdk.log" file that is stored in the same directory as the "global-settings.xdk" file to the Intel XDK forum. Please ATTACH the xdk.log file to your post using the "Attach Files to Post" link below the forum edit window.

Is Intel XDK an open source project? How can I contribute to the Intel XDK community?

No, It is not an open source project. However, it utilizes many open source components that are then assembled into Intel XDK. While you cannot contribute directly to the Intel XDK integration effort, you can contribute to the many open source components that make up Intel XDK.

The following open source components are the major elements that are being used by Intel XDK:

• Node-Webkit
• Chromium
• Ripple* emulator
• Brackets* editor
• Weinre* remote debugger
• Crosswalk*
• Cordova*
• App Framework*

How do I configure Intel XDK to use 9 patch png for Android* apps splash screen?

Intel XDK does support the use of 9 patch png for Android* apps splash screen. You can read up more at https://software.intel.com/en-us/xdk/articles/android-splash-screens-using-nine-patch-png on how to create a 9 patch png image and link to an Intel XDK sample using 9 patch png images.

How do I stop AVG from popping up the "General Behavioral Detection" window when Intel XDK is launched?

You can try adding nw.exe as the app that needs an exception in AVG.

What do I specify for "App ID" in Intel XDK under Build Settings?

Your app ID uniquely identifies your app. For example, it can be used to identify your app within Apple’s application services allowing you to use things like in-app purchasing and push notifications.

Here are some useful articles on how to create an App ID:

• iOS* App
• Android* App
• Windows* Phone 8 App

Is it possible to modify the Android Manifest or iOS plist file with the Intel XDK?

You cannot modify the AndroidManifest.xml file directly with our build system, as it only exists in the cloud. However, you may do so by creating a dummy plugin that only contains a plugin.xml file containing directives that can be used to add lines to the AndroidManifest.xml file during the build process. In essence, you add lines to the AndroidManifest.xml file via a local plugin.xml file. Here is an example of a plugin that does just that:

<?xml version="1.0" encoding="UTF-8"?><plugin xmlns="http://apache.org/cordova/ns/plugins/1.0" id="my-custom-intents-plugin" version="1.0.0"><name>My Custom Intents Plugin</name><description>Add Intents to the AndroidManifest.xml</description><license>MIT</license><engines><engine name="cordova" version=">=3.0.0" /></engines><!-- android --><platform name="android"><config-file target="AndroidManifest.xml" parent="/manifest/application"><activity android:configChanges="orientation|keyboardHidden|keyboard|screenSize|locale" android:label="@string/app_name" android:launchMode="singleTop" android:name="testa" android:theme="@android:style/Theme.Black.NoTitleBar"><intent-filter><action android:name="android.intent.action.SEND" /><category android:name="android.intent.category.DEFAULT" /><data android:mimeType="*/*" /></intent-filter></activity></config-file></platform></plugin>

You can inspect the AndroidManifest.xml created in an APK, using apktool with the following command line:

$ apktool d my-app.apk
$ cd my-app
$ more AndroidManifest.xml

This technique exploits the config-file element that is described in the Cordova Plugin Specification docs and can also be used to add lines to iOS plist files. See the Cordova plugin documentation link for additional details.

Here is an example of such a plugin for modifying the iOS plist file, specifically for adding a BIS key to the plist file:

<?xml version="1.0" encoding="UTF-8"?><plugin
    xmlns="http://apache.org/cordova/ns/plugins/1.0"
    id="my-custom-bis-plugin"
    version="0.0.2"><name>My Custom BIS Plugin</name><description>Add BIS info to iOS plist file.</description><license>BSD-3</license><preference name="BIS_KEY" /><engines><engine name="cordova" version=">=3.0.0" /></engines><!-- ios --><platform name="ios"><config-file target="*-Info.plist" parent="CFBundleURLTypes"><array><dict><key>ITSAppUsesNonExemptEncryption</key><true/><key>ITSEncryptionExportComplianceCode</key><string>$BIS_KEY</string></dict></array></config-file></platform></plugin>

And see this forum thread > https://software.intel.com/en-us/forums/intel-xdk/topic/680309< for an example of how to customize the OneSignal plugin's notification sound, in an Android app, by way of using a simple custom Cordova plugin. The same technique can be applied to adding custom icons and other assets to your project.

How can I share my Intel XDK app build?

You can send a link to your project via an email invite from your project settings page. However, a login to your account is required to access the file behind the link. Alternatively, you can download the build from the build page, onto your workstation, and push that built image to some location from which you can send a link to that image.

Why does my iOS build fail when I am able to test it successfully on a device and the emulator?

Common reasons include:

• Your App ID specified in the project settings do not match the one you specified in Apple's developer portal.
• The provisioning profile does not match the cert you uploaded. Double check with Apple's developer site that you are using the correct and current distribution cert and that the provisioning profile is still active. Download the provisioning profile again and add it to your project to confirm.
• In Project Build Settings, your App Name is invalid. It should be modified to include only alpha, space and numbers.

How do I add multiple domains in Domain Access?

Here is the primary doc source for that feature.

If you need to insert multiple domain references, then you will need to add the extra references in the intelxdk.config.additions.xml file. This StackOverflow entry provides a basic idea and you can see the intelxdk.config.*.xml files that are automatically generated with each build for the <access origin="xxx" /> line that is generated based on what you provide in the "Domain Access" field of the "Build Settings" panel on the Project Tab.

How do I build more than one app using the same Apple developer account?

On Apple developer, create a distribution certificate using the "iOS* Certificate Signing Request" key downloaded from Intel XDK Build tab only for the first app. For subsequent apps, reuse the same certificate and import this certificate into the Build tab like you usually would.

How do I include search and spotlight icons as part of my app?

Please refer to this article in the Intel XDK documentation. Create anintelxdk.config.additions.xml file in your top level directory (same location as the otherintelxdk.*.config.xml files) and add the following lines for supporting icons in Settings and other areas in iOS*.

<!-- Spotlight Icon --><icon platform="ios" src="res/ios/icon-40.png" width="40" height="40" /><icon platform="ios" src="res/ios/icon-40@2x.png" width="80" height="80" /><icon platform="ios" src="res/ios/icon-40@3x.png" width="120" height="120" /><!-- iPhone Spotlight and Settings Icon --><icon platform="ios" src="res/ios/icon-small.png" width="29" height="29" /><icon platform="ios" src="res/ios/icon-small@2x.png" width="58" height="58" /><icon platform="ios" src="res/ios/icon-small@3x.png" width="87" height="87" /><!-- iPad Spotlight and Settings Icon --><icon platform="ios" src="res/ios/icon-50.png" width="50" height="50" /><icon platform="ios" src="res/ios/icon-50@2x.png" width="100" height="100" />

For more information related to these configurations, visit http://cordova.apache.org/docs/en/3.5.0/config_ref_images.md.html#Icons%20and%20Splash%20Screens.

For accurate information related to iOS icon sizes, visit https://developer.apple.com/library/ios/documentation/UserExperience/Conceptual/MobileHIG/IconMatrix.html

NOTE: The iPhone 6 icons will only be available if iOS* 7 or 8 is the target.

Cordova iOS* 8 support JIRA tracker: https://issues.apache.org/jira/browse/CB-7043

Does Intel XDK support Modbus TCP communication?

No, since Modbus is a specialized protocol, you need to write either some JavaScript* or native code (in the form of a plugin) to handle the Modbus transactions and protocol.

How do I sign an Android* app using an existing keystore?

New with release 3088 of the Intel XDK, you may now import your existing keystore into Intel XDK using the new certificate manager that is built into the Intel XDK. Please read the initial paragraphs of Managing Certificates for your Intel XDK Account and the section titled "Import an Android Certificate Keystore" in that document, for details regarding how to do this.

If the above fails, please send an email to html5tools@intel.com requesting help. It is important that you send that email from the email address associated with your Intel XDK account.

How do I build separately for different Android* versions?

Under the Projects Panel, you can select the Target Android* version under the Build Settings collapsible panel. You can change this value and build your application multiple times to create numerous versions of your application that are targeted for multiple versions of Android*.

How do I display the 'Build App Now' button if my display language is not English?

If your display language is not English and the 'Build App Now' button is proving to be troublesome, you may change your display language to English which can be downloaded by a Windows* update. Once you have installed the English language, proceed to Control Panel > Clock, Language and Region > Region and Language > Change Display Language.

How do I update my Intel XDK version?

When an Intel XDK update is available, an Update Version dialog box lets you download the update. After the download completes, a similar dialog lets you install it. If you did not download or install an update when prompted (or on older versions), click the package icon next to the orange (?) icon in the upper-right to download or install the update. The installation removes the previous Intel XDK version.

How do I import my existing HTML5 app into the Intel XDK?

If your project contains an Intel XDK project file (<project-name>.xdk) you should use the "Open an Intel XDK Project" option located at the bottom of the Projects List on the Projects tab (lower left of the screen, round green "eject" icon, on the Projects tab). This would be the case if you copied an existing Intel XDK project from another system or used a tool that exported a complete Intel XDK project.

If your project does not contain an Intel XDK project file (<project-name>.xdk) you must "import" your code into a new Intel XDK project. To import your project, use the "Start a New Project" option located at the bottom of the Projects List on the Projects tab (lower left of the screen, round blue "plus" icon, on theProjects tab). This will open the "Samples, Demos and Templates" page, which includes an option to "Import Your HTML5 Code Base." Point to the root directory of your project. The Intel XDK will attempt to locate a file named index.html in your project and will set the "Source Directory" on the Projects tab to point to the directory that contains this file.

If your imported project did not contain an index.html file, your project may be unstable. In that case, it is best to delete the imported project from the Intel XDK Projects tab ("x" icon in the upper right corner of the screen), rename your "root" or "main" html file to index.html and import the project again. Several components in the Intel XDK depend on this assumption that the main HTML file in your project is named index.hmtl. See Introducing Intel® XDK Development Tools for more details.

It is highly recommended that your "source directory" be located as a sub-directory inside your "project directory." This insures that non-source files are not included as part of your build package when building your application. If the "source directory" and "project directory" are the same it results in longer upload times to the build server and unnecessarily large application executable files returned by the build system. See the following images for the recommended project file layout.

I am unable to login to App Preview with my Intel XDK password.

On some devices you may have trouble entering your Intel XDK login password directly on the device in the App Preview login screen. In particular, sometimes you may have trouble with the first one or two letters getting lost when entering your password.

Try the following if you are having such difficulties:

• Reset your password, using the Intel XDK, to something short and simple.

• Confirm that this new short and simple password works with the XDK (logout and login to the Intel XDK).

• Confirm that this new password works with the Intel Developer Zone login.

• Make sure you have the most recent version of Intel App Preview installed on your devices. Go to the store on each device to confirm you have the most recent copy of App Preview installed.

• Try logging into Intel App Preview on each device with this short and simple password. Check the "show password" box so you can see your password as you type it.

If the above works, it confirms that you can log into your Intel XDK account from App Preview (because App Preview and the Intel XDK go to the same place to authenticate your login). When the above works, you can go back to the Intel XDK and reset your password to something else, if you do not like the short and simple password you used for the test.

If you are having trouble logging into any pages on the Intel web site (including the Intel XDK forum), please see the Intel Sign In FAQ for suggestions and contact info. That login system is the backend for the Intel XDK login screen.

How do I completely uninstall the Intel XDK from my system?

Take the following steps to completely uninstall the XDK from your Windows system:

• From the Windows Control Panel, remove the Intel XDK, using the Windows uninstall tool.

• Then:
  > cd %LocalAppData%\Intel\XDK
> del *.* /s/q

• Then:
  > cd %LocalAppData%\XDK
> copy global-settings.xdk %UserProfile%
> del *.* /s/q
> copy %UserProfile%\global-settings.xdk .

• Then:
  -- Goto xdk.intel.com and select the download link.
  -- Download and install the new XDK.

To do the same on a Linux or Mac system:

• On a Linux machine, run the uninstall script, typically /opt/intel/XDK/uninstall.sh.
   
• Remove the directory into which the Intel XDK was installed.
  -- Typically /opt/intel or your home (~) directory on a Linux machine.
  -- Typically in the /Applications/Intel XDK.app directory on a Mac.
   
• Then:
  $ find ~ -name global-settings.xdk
$ cd <result-from-above> (for example ~/Library/Application Support/XDK/ on a Mac)
$ cp global-settings.xdk ~
$ rm -Rf *
$ mv ~/global-settings.xdk .
   
• Then:
  -- Goto xdk.intel.com and select the download link.
  -- Download and install the new XDK.

Is there a tool that can help me highlight syntax issues in Intel XDK?

Yes, you can use the various linting tools that can be added to the Brackets editor to review any syntax issues in your HTML, CSS and JS files. Go to the "File > Extension Manager..." menu item and add the following extensions: JSHint, CSSLint, HTMLHint, XLint for Intel XDK. Then, review your source files by monitoring the small yellow triangle at the bottom of the edit window (a green check mark indicates no issues).

How do I delete built apps and test apps from the Intel XDK build servers?

You can manage them by logging into: https://appcenter.html5tools-software.intel.com/csd/controlpanel.aspx. This functionality will eventually be available within Intel XDK after which access to app center will be removed.

I need help with the App Security API plugin; where do I find it?

Visit the primary documentation book for the App Security API and see this forum post for some additional details.

When I install my app or use the Debug tab Avast antivirus flags a possible virus, why?

If you are receiving a "Suspicious file detected - APK:CloudRep [Susp]" message from Avast anti-virus installed on your Android device it is due to the fact that you are side-loading the app (or the Intel XDK Debug modules) onto your device (using a download link after building or by using the Debug tab to debug your app), or your app has been installed from an "untrusted" Android store. See the following official explanation from Avast:

Your application was flagged by our cloud reputation system. "Cloud rep" is a new feature of Avast Mobile Security, which flags apks when the following conditions are met:

1. The file is not prevalent enough; meaning not enough users of Avast Mobile Security have installed your APK.
2. The source is not an established market (Google Play is an example of an established market).

If you distribute your app using Google Play (or any other trusted market) your users should not see any warning from Avast.

Following are some of the Avast anti-virus notification screens you might see on your device. All of these are perfectly normal, they are due to the fact that you must enable the installation of "non-market" apps in order to use your device for debug and the App IDs associated with your never published app or the custom debug modules that the Debug tab in the Intel XDK builds and installs on your device will not be found in a "established" (aka "trusted") market, such as Google Play.

If you choose to ignore the "Suspicious app activity!" threat you will not receive a threat for that debug module any longer. It will show up in the Avast 'ignored issues' list. Updates to an existing, ignored, custom debug module should continue to be ignored by Avast. However, new custom debug modules (due to a new project App ID or a new version of Crosswalk selected in your project's Build Settings) will result in a new warning from the Avast anti-virus tool.

How do I add a Brackets extension to the editor that is part of the Intel XDK?

The number of Brackets extensions that are provided in the built-in edition of the Brackets editor are limited to insure stability of the Intel XDK product. Not all extensions are compatible with the edition of Brackets that is embedded within the Intel XDK. Adding incompatible extensions can cause the Intel XDK to quit working.

Despite this warning, there are useful extensions that have not been included in the editor and which can be added to the Intel XDK. Adding them is temporary, each time you update the Intel XDK (or if you reinstall the Intel XDK) you will have to "re-add" your Brackets extension. To add a Brackets extension, use the following procedure:

• exit the Intel XDK
• download a ZIP file of the extension you wish to add
• on Windows, unzip the extension here:
  %LocalAppData%\Intel\XDK\xdk\brackets\b\extensions\dev
• on Mac OS X, unzip the extension here:
  /Applications/Intel\ XDK.app/Contents/Resources/app.nw/brackets/b/extensions/dev
• start the Intel XDK

Note that the locations given above are subject to change with new releases of the Intel XDK.

Why does my app or game require so many permissions on Android when built with the Intel XDK?

When you build your HTML5 app using the Intel XDK for Android or Android-Crosswalk you are creating a Cordova app. It may seem like you're not building a Cordova app, but you are. In order to package your app so it can be distributed via an Android store and installed on an Android device, it needs to be built as a hybrid app. The Intel XDK uses Cordova to create that hybrid app.

A pure Cordova app requires the NETWORK permission, it's needed to "jump" between your HTML5 environment and the native Android environment. Additional permissions will be added by any Cordova plugins you include with your application; which permissions are includes are a function of what that plugin does and requires.

Crosswalk for Android builds also require the NETWORK permission, because the Crosswalk image built by the Intel XDK includes support for Cordova. In addition, current versions of Crosswalk (12 and 14 at the time this FAQ was written)also require NETWORK STATE and WIFI STATE. There is an extra permission in some versions of Crosswalk (WRITE EXTERNAL STORAGE) that is only needed by the shared model library of Crosswalk, we have asked the Crosswalk project to remove this permission in a future Crosswalk version.

If you are seeing more than the following five permissions in your XDK-built Crosswalk app:

• android.permission.INTERNET
• android.permission.ACCESS_NETWORK_STATE
• android.permission.ACCESS_WIFI_STATE
• android.permission.INTERNET
• android.permission.WRITE_EXTERNAL_STORAGE

then you are seeing permissions that have been added by some plugins. Each plugin is different, so there is no hard rule of thumb. The two "default" core Cordova plugins that are added by the Intel XDK blank templates (device and splash screen) do not require any Android permissions.

BTW: the permission list above comes from a Crosswalk 14 build. Crosswalk 12 builds do not included the last permission; it was added when the Crosswalk project introduced the shared model library option, which started with Crosswalk 13 (the Intel XDK does not support 13 builds).

How do I make a copy of an existing Intel XDK project?

If you just need to make a backup copy of an existing project, and do not plan to open that backup copy as a project in the Intel XDK, do the following:

• Exit the Intel XDK.
• Copy the entire project directory:
  • on Windows, use File Explorer to "right-click" and "copy" your project directory, then "right-click" and "paste"
  • on Mac use Finder to "right-click" and then "duplicate" your project directory
  • on Linux, open a terminal window, "cd" to the folder that contains your project, and type "cp -a old-project/ new-project/" at the terminal prompt (where "old-project/" is the folder name of your existing project that you want to copy and "new-project/" is the name of the new folder that will contain a copy of your existing project)

If you want to use an existing project as the starting point of a new project in the Intel XDK. The process described below will insure that the build system does not confuse the ID in your old project with that stored in your new project. If you do not follow the procedure below you will have multiple projects using the same project ID (a special GUID that is stored inside the Intel XDK <project-name>.xdk file in the root directory of your project). Each project in your account must have a unique project ID.

• Exit the Intel XDK.
• Make a copy of your existing project using the process described above.
• Inside the new project that you made (that is, your new copy of your old project), make copies of the <project-name>.xdk file and <project-name>.xdke files and rename those copies to something like project-new.xdk and project-new.xdke (anything you like, just something different than the original project name, preferably the same name as the new project folder in which you are making this new project).
• Using a TEXT EDITOR (only) (such as Notepad or Sublime or Brackets or some other TEXT editor), open your new "project-new.xdk" file (whatever you named it) and find the projectGuid line, it will look something like this:
  "projectGuid": "a863c382-ca05-4aa4-8601-375f9f209b67",
• Change the "GUID" to all zeroes, like this: "00000000-0000-0000-000000000000"
• Save the modified "project-new.xdk" file.
• Open the Intel XDK.
• Goto the Projects tab.
• Select "Open an Intel XDK Project" (the green button at the bottom left of the Projects tab).
• To open this new project, locate the new "project-new.xdk" file inside the new project folder you copied above.
• Don't forget to change the App ID in your new project. This is necessary to avoid conflicts with the project you copied from, in the store and when side-loading onto a device.

My project does not include a www folder. How do I fix it so it includes a www or source directory?

The Intel XDK HTML5 and Cordova project file structures are meant to mimic a standard Cordova project. In a Cordova (or PhoneGap) project there is a subdirectory (or folder) named www that contains all of the HTML5 source code and asset files that make up your application. For best results, it is advised that you follow this convention, of putting your source inside a "source directory" inside of your project folder.

This most commonly happens as the result of exporting a project from an external tool, such as Construct2, or as the result of importing an existing HTML5 web app that you are converting into a hybrid mobile application (eg., an Intel XDK Corodova app). If you would like to convert an existing Intel XDK project into this format, follow the steps below:

• Exit the Intel XDK.
• Copy the entire project directory:
  • on Windows, use File Explorer to "right-click" and "copy" your project directory, then "right-click" and "paste"
  • on Mac use Finder to "right-click" and then "duplicate" your project directory
  • on Linux, open a terminal window, "cd" to the folder that contains your project, and type "cp -a old-project/ new-project/" at the terminal prompt (where "old-project/" is the folder name of your existing project that you want to copy and "new-project/" is the name of the new folder that will contain a copy of your existing project)
• Create a "www" directory inside the new duplicate project you just created above.
• Move your index.html and other source and asset files to the "www" directory you just created -- this is now your "source" directory, located inside your "project" directory (do not move the <project-name>.xdk and xdke files and any intelxdk.config.*.xml files, those must stay in the root of the project directory)
• Inside the new project that you made above (by making a copy of the old project), rename the <project-name>.xdk file and <project-name>.xdke files to something like project-copy.xdk and project-copy.xdke (anything you like, just something different than the original project, preferably the same name as the new project folder in which you are making this new project).
• Using a TEXT EDITOR (only) (such as Notepad or Sublime or Brackets or some other TEXT editor), open the new "project-copy.xdk" file (whatever you named it) and find the line named projectGuid, it will look something like this:
  "projectGuid": "a863c382-ca05-4aa4-8601-375f9f209b67",
• Change the "GUID" to all zeroes, like this: "00000000-0000-0000-000000000000"
• A few lines down find: "sourceDirectory": "",
• Change it to this: "sourceDirectory": "www",
• Save the modified "project-copy.xdk" file.
• Open the Intel XDK.
• Goto the Projects tab.
• Select "Open an Intel XDK Project" (the green button at the bottom left of the Projects tab).
• To open this new project, locate the new "project-copy.xdk" file inside the new project folder you copied above.

Can I install more than one copy of the Intel XDK onto my development system?

Yes, you can install more than one version onto your development system. However, you cannot run multiple instances of the Intel XDK at the same time. Be aware that new releases sometimes change the project file format, so it is a good idea, in these cases, to make a copy of your project if you need to experiment with a different version of the Intel XDK. See the instructions in a FAQ entry above regarding how to make a copy of your Intel XDK project.

Follow the instructions in this forum post to install more than one copy of the Intel XDK onto your development system.

On Apple OS X* and Linux* systems, does the Intel XDK need the OpenSSL* library installed?

Yes. Several features of the Intel XDK require the OpenSSL library, which typically comes pre-installed on Linux and OS X systems. If the Intel XDK reports that it could not find libssl, go to https://www.openssl.org to download and install it.

I have a web application that I would like to distribute in app stores without major modifications. Is this possible using the Intel XDK?

Yes, if you have a true web app or “client app” that only uses HTML, CSS and JavaScript, it is usually not too difficult to convert it to a Cordova hybrid application (this is what the Intel XDK builds when you create an HTML5 app). If you rely heavily on PHP or other server scripting languages embedded in your pages you will have more work to do. Because your Cordova app is not associated with a server, you cannot rely on server-based programming techniques; instead, you must rewrite any such code to user RESTful APIs that your app interacts with using, for example, AJAX calls.

What is the best training approach to using the Intel XDK for a newbie?

First, become well-versed in the art of client web apps, apps that rely only on HTML, CSS and JavaScript and utilize RESTful APIs to talk to network services. With that you will have mastered 80% of the problem. After that, it is simply a matter of understanding how Cordova plugins are able to extend the JavaScript API for access to features of the platform. For HTML5 training there are many sites providing tutorials. It may also help to read Five Useful Tips on Getting Started Building Cordova Mobile Apps with the Intel XDK, which will help you understand some of the differences between developing for a traditional server-based environment and developing for the Intel XDK hybrid Cordova app environment.

What is the best platform to start building an app with the Intel XDK? And what are the important differences between the Android, iOS and other mobile platforms?

There is no one most important difference between the Android, iOS and other platforms. It is important to understand that the HTML5 runtime engine that executes your app on each platform will vary as a function of the platform. Just as there are differences between Chrome and Firefox and Safari and Internet Explorer, there are differences between iOS 9 and iOS 8 and Android 4 and Android 5, etc. Android has the most significant differences between vendors and versions of Android. This is one of the reasons the Intel XDK offers the Crosswalk for Android build option, to normalize and update the Android issues.

In general, if you can get your app working well on Android (or Crosswalk for Android) first you will generally have fewer issues to deal with when you start to work on the iOS and Windows platforms. In addition, the Android platform has the most flexible and useful debug options available, so it is the easiest platform to use for debugging and testing your app.

Is my password encrypted and why is it limited to fifteen characters?

Yes, your password is stored encrypted and is managed by https://signin.intel.com. Your Intel XDK userid and password can also be used to log into the Intel XDK forum as well as the Intel Developer Zone. the Intel XDK does not store nor does it manage your userid and password.

The rules regarding allowed userids and passwords are answered on this Sign In FAQ page, where you can also find help on recovering and changing your password.

Why does the Intel XDK take a long time to start on Linux or Mac?

...and why am I getting this error message? "Attempt to contact authentication server is taking a long time. You can wait, or check your network connection and try again."

At startup, the Intel XDK attempts to automatically determine the proxy settings for your machine. Unfortunately, on some system configurations it is unable to reliably detect your system proxy settings. As an example, you might see something like this image when starting the Intel XDK.

On some systems you can get around this problem by setting some proxy environment variables and then starting the Intel XDK from a command-line that includes those configured environment variables. To set those environment variables, similar to the following:

$ export no_proxy="localhost,127.0.0.1/8,::1"
$ export NO_PROXY="localhost,127.0.0.1/8,::1"
$ export http_proxy=http://proxy.mydomain.com:123/
$ export HTTP_PROXY=http://proxy.mydomain.com:123/
$ export https_proxy=http://proxy.mydomain.com:123/
$ export HTTPS_PROXY=http://proxy.mydomain.com:123/

IMPORTANT! The name of your proxy server and the port (or ports) that your proxy server requires will be different than those shown in the example above. Please consult with your IT department to find out what values are appropriate for your site. Intel has no way of knowing what configuration is appropriate for your network.

If you use the Intel XDK in multiple locations (at work and at home), you may have to change the proxy settings before starting the Intel XDK after switching to a new network location. For example, many work networks use a proxy server, but most home networks do not require such a configuration. In that case, you need to be sure to "unset" the proxy environment variables before starting the Intel XDK on a non-proxy network.

After you have successfully configured your proxy environment variables, you can start the Intel XDK manually, from the command-line.

On a Mac, where the Intel XDK is installed in the default location, type the following (from a terminal window that has the above environment variables set):

$ open /Applications/Intel\ XDK.app/

On a Linux machine, assuming the Intel XDK has been installed in the ~/intel/XDK directory, type the following (from a terminal window that has the above environment variables set):

$ ~/intel/XDK/xdk.sh &

In the Linux case, you will need to adjust the directory name that points to the xdk.sh file in order to start. The example above assumes a local install into the ~/intel/XDK directory. Since Linux installations have more options regarding the installation directory, you will need to adjust the above to suit your particular system and install directory.

How do I generate a P12 file on a Windows machine?

See these articles:

• How to Create an iOS Provisioning Profile and P12 with Windows
• Building an iOS signing key for PhoneGap in Windows

How do I change the default dir for creating new projects in the Intel XDK?

You can change the default new project location manually by modifying a field in the global-settings.xdk file. Locate the global-settings.xdk file on your system (the precise location varies as a function of the OS) and find this JSON object inside that file:

"projects-tab": {"defaultPath": "/Users/paul/Documents/XDK","LastSortType": "descending|Name","lastSortType": "descending|Opened","thirdPartyDisclaimerAcked": true
  },

The example above came from a Mac. On a Mac the global-settings.xdk file is located in the "~/Library/Application Support/XDK" directory.

On a Windows machine the global-settings.xdk file is normally found in the "%LocalAppData%\XDK" directory. The part you are looking for will look something like this:

"projects-tab": {"thirdPartyDisclaimerAcked": false,"LastSortType": "descending|Name","lastSortType": "descending|Opened","defaultPath": "C:\\Users\\paul/Documents"
  },

Obviously, it's the defaultPath part you want to change.

BE CAREFUL WHEN YOU EDIT THE GLOBAL-SETTINGS.XDK FILE!! You've been warned...

Make sure the result is proper JSON when you are done, or it may cause your XDK to cough and hack loudly. Make a backup copy of global-settings.xdk before you start, just in case.

Where I can find recent and upcoming webinars list?

• Upcoming webinars list.
• Recent webinars list.

How can I change the email address associated with my Intel XDK login?

Login to the Intel Developer Zone with your Intel XDK account userid and password and then locate your "account dashboard." Click the "pencil icon" next to your name to open the "Personal Profile" section of your account, where you can edit your "Name & Contact Info," including the email address associated with your account, under the "Private" section of your profile.

What network addresses must I enable in my firewall to insure the Intel XDK will work on my restricted network?

Normally, access to the external servers that the Intel XDK uses is handled automatically by your proxy server. However, if you are working in an environment that has restricted Internet access and you need to provide your IT department with a list of URLs that you need access to in order to use the Intel XDK, then please provide them with the following list of domain names:

• appcenter.html5tools-software.intel.com (for communication with the build servers)
• s3.amazonaws.com (for downloading sample apps and built apps)
• download.xdk.intel.com (for getting XDK updates)
• xdk-feed-proxy.html5tools-software.intel.com (for receiving the tweets in the upper right corner of the XDK)
• signin.intel.com (for logging into the XDK)
• sfederation.intel.com (for logging into the XDK)

Normally this should be handled by your network proxy (if you're on a corporate network) or should not be an issue if you are working on a typical home network.

I cannot create a login for the Intel XDK, how do I create a userid and password to use the Intel XDK?

If you have downloaded and installed the Intel XDK but are having trouble creating a login, you can create the login outside the Intel XDK. To do this, go to the Intel Developer Zone and push the "Join Today" button. After you have created your Intel Developer Zone login you can return to the Intel XDK and use that userid and password to login to the Intel XDK. This same userid and password can also be used to login to the Intel XDK forum.

Installing the Intel XDK on Windows fails with a "Package signature verification failed." message.

If you receive a "Package signature verification failed" message (see image below) when installing the Intel XDK on your system, it is likely due to one of the following two reasons:

• Your system does not have a properly installed "root certificate" file, which is needed to confirm that the install package is good.
• The install package is corrupt and failed the verification step.

The first case can happen if you are attempting to install the Intel XDK on an unsupported version of Windows. The Intel XDK is only supported on Microsoft Windows 7 and higher. If you attempt to install on Windows Vista (or earlier) you may see this verification error. The workaround is to install the Intel XDK on a Windows 7 or greater machine.

The second case is likely due to a corruption of the install package during download or due to tampering. The workaround is to re-download the install package and attempt another install.

If you are installing on a Windows 7 (or greater) machine and you see this message it is likely due to a missing or bad root certificate on your system. To fix this you may need to start the "Certificate Propagation" service. Open the Windows "services.msc" panel and then start the "Certificate Propagation" service. Additional links related to this problem can be found here > https://technet.microsoft.com/en-us/library/cc754841.aspx

See this forum thread for additional help regarding this issue > https://software.intel.com/en-us/forums/intel-xdk/topic/603992

Troubles installing the Intel XDK on a Linux or Ubuntu system, which option should I choose?

Choose the local user option, not root or sudo, when installing the Intel XDK on your Linux or Ubuntu system. This is the most reliable and trouble-free option and is the default installation option. This will insure that the Intel XDK has all the proper permissions necessary to execute properly on your Linux system. The Intel XDK will be installed in a subdirectory of your home (~) directory.

Inactive account/ login issue/ problem updating an APK in store, How do I request account transfer?

As of June 26, 2015 we migrated all of Intel XDK accounts to the more secure intel.com login system (the same login system you use to access this forum).

We have migrated nearly all active users to the new login system. Unfortunately, there are a few active user accounts that we could not automatically migrate to intel.com, primarily because the intel.com login system does not allow the use of some characters in userids that were allowed in the old login system.

If you have not used the Intel XDK for a long time prior to June 2015, your account may not have been automatically migrated. If you own an "inactive" account it will have to be manually migrated -- please try logging into the Intel XDK with your old userid and password, to determine if it no longer works. If you find that you cannot login to your existing Intel XDK account, and still need access to your old account, please send a message to html5tools@intel.com and include your userid and the email address associated with that userid, so we can guide you through the steps required to reactivate your old account.

Alternatively, you can create a new Intel XDK account. If you have submitted an app to the Android store from your old account you will need access to that old account to retrieve the Android signing certificates in order to upgrade that app on the Android store; in that case, send an email to html5tools@intel.com with your old account username and email and new account information.

Connection Problems? -- Intel XDK SSL certificates update

On January 26, 2016 we updated the SSL certificates on our back-end systems to SHA2 certificates. The existing certificates were due to expire in February of 2016. We have also disabled support for obsolete protocols.

If you are experiencing persistent connection issues (since Jan 26, 2016), please post a problem report on the forum and include in your problem report:

• the operation that failed
• the version of your XDK
• the version of your operating system
• your geographic region
• and a screen capture

How do I resolve build failure: "libpng error: Not a PNG file"?  

f you are experiencing build failures with CLI 5 Android builds, and the detailed error log includes a message similar to the following:

Execution failed for task ':mergeArmv7ReleaseResources'.> Error: Failed to run command: /Developer/android-sdk-linux/build-tools/22.0.1/aapt s -i .../platforms/android/res/drawable-land-hdpi/screen.png -o .../platforms/android/build/intermediates/res/armv7/release/drawable-land-hdpi-v4/screen.png

Error Code: 42

Output: libpng error: Not a PNG file

You need to change the format of your icon and/or splash screen images to PNG format.

The error message refers to a file named "screen.png" -- which is what each of your splash screens were renamed to before they were moved into the build project resource directories. Unfortunately, JPG images were supplied for use as splash screen images, not PNG images. So the files were renamed and found by the build system to be invalid.

Convert your splash screen images to PNG format. Renaming JPG images to PNG will not work! You must convert your JPG images into PNG format images using an appropriate image editing tool. The Intel XDK does not provide any such conversion tool.

Beginning with Cordova CLI 5, all icons and splash screen images must be supplied in PNG format. This applies to all supported platforms. This is an undocumented "new feature" of the Cordova CLI 5 build system that was implemented by the Apache Cordova project.

Why do I get a "Parse Error" when I try to install my built APK on my Android device?

Because you have built an "unsigned" Android APK. You must click the "signed" box in the Android Build Settings section of the Projects tab if you want to install an APK on your device. The only reason you would choose to create an "unsigned" APK is if you need to sign it manually. This is very rare and not the normal situation.

My converted legacy keystore does not work. Google Play is rejecting my updated app.

The keystore you converted when you updated to 3088 (now 3240 or later) is the same keystore you were using in 2893. When you upgraded to 3088 (or later) and "converted" your legacy keystore, you re-signed and renamed your legacy keystore and it was transferred into a database to be used with the Intel XDK certificate management tool. It is still the same keystore, but with an alias name and password assigned by you and accessible directly by you through the Intel XDK.

If you kept the converted legacy keystore in your account following the conversion you can download that keystore from the Intel XDK for safe keeping (do not delete it from your account or from your system). Make sure you keep track of the new password(s) you assigned to the converted keystore.

There are two problems we have experienced with converted legacy keystores at the time of the 3088 release (April, 2016):

• Using foreign (non-ASCII) characters in the new alias name and passwords were being corrupted.
• Final signing of your APK by the build system was being done with RSA256 rather than SHA1.

Both of the above items have been resolved and should no longer be an issue.

If you are currently unable to complete a build with your converted legacy keystore (i.e., builds fail when you use the converted legacy keystore but they succeed when you use a new keystore) the first bullet above is likely the reason your converted keystore is not working. In that case we can reset your converted keystore and give you the option to convert it again. You do this by requesting that your legacy keystore be "reset" by filling out this form. For 100% surety during that second conversion, use only 7-bit ASCII characters in the alias name you assign and for the password(s) you assign.

IMPORTANT: using the legacy certificate to build your Android app is ONLY necessary if you have already published an app to an Android store and need to update that app. If you have never published an app to an Android store using the legacy certificate you do not need to concern yourself with resetting and reconverting your legacy keystore. It is easier, in that case, to create a new Android keystore and use that new keystore.

If you ARE able to successfully build your app with the converted legacy keystore, but your updated app (in the Google store) does not install on some older Android 4.x devices (typically a subset of Android 4.0-4.2 devices), the second bullet cited above is likely the reason for the problem. The solution, in that case, is to rebuild your app and resubmit it to the store (that problem was a build-system problem that has been resolved).

How can I have others beta test my app using Intel App Preview?

Apps that you sync to your Intel XDK account, using the Test tab's green "Push Files" button, can only be accessed by logging into Intel App Preview with the same Intel XDK account credentials that you used to push the files to the cloud. In other words, you can only download and run your app for testing with Intel App Preview if you log into the same account that you used to upload that test app. This restriction applies to downloading your app into Intel App Preview via the "Server Apps" tab, at the bottom of the Intel App Preview screen, or by scanning the QR code displayed on the Intel XDK Test tab using the camera icon in the upper right corner of Intel App Preview.

If you want to allow others to test your app, using Intel App Preview, it means you must use one of two options:

• give them your Intel XDK userid and password
• create an Intel XDK "test account" and provide your testers with that userid and password

For security sake, we highly recommend you use the second option (create an Intel XDK "test account"). 

A "test account" is simply a second Intel XDK account that you do not plan to use for development or builds. Do not use the same email address for your "test account" as you are using for your main development account. You should use a "throw away" email address for that "test account" (an email address that you do not care about).

Assuming you have created an Intel XDK "test account" and have instructed your testers to download and install Intel App Preview; have provided them with your "test account" userid and password; and you are ready to have them test:

• sign out of your Intel XDK "development account" (using the little "man" icon in the upper right)
• sign into your "test account" (again, using the little "man" icon in the Intel XDK toolbar)
• make sure you have selected the project that you want users to test, on the Projects tab
• goto the Test tab
• make sure "MOBILE" is selected (upper left of the Test tab)
• push the green "PUSH FILES" button on the Test tab
• log out of your "test account"
• log into your development account

Then, tell your beta testers to log into Intel App Preview with your "test account" credentials and instruct them to choose the "Server Apps" tab at the bottom of the Intel App Preview screen. From there they should see the name of the app you synced using the Test tab and can simply start it by touching the app name (followed by the big blue and white "Launch This App" button). Staring the app this way is actually easier than sending them a copy of the QR code. The QR code is very dense and is hard to read with some devices, dependent on the quality of the camera in their device.

Note that when running your test app inside of Intel App Preview they cannot test any features associated with third-party plugins, only core Cordova plugins. Thus, you need to insure that those parts of your apps that depend on non-core Cordova plugins have been disabled or have exception handlers to prevent your app from either crashing or freezing.

I'm having trouble making Google Maps work with my Intel XDK app. What can I do?

There are many reasons that can cause your attempt to use Google Maps to fail. Mostly it is due to the fact that you need to download the Google Maps API (JavaScript library) at runtime to make things work. However, there is no guarantee that you will have a good network connection, so if you do it the way you are used to doing it, in a browser...

<script src="https://maps.googleapis.com/maps/api/js?key=API_KEY&sensor=true"></script>

...you may get yourself into trouble, in an Intel XDK Cordova app. See Loading Google Maps in Cordova the Right Way for an excellent tutorial on why this is a problem and how to deal with it. Also, it may help to read Five Useful Tips on Getting Started Building Cordova Mobile Apps with the Intel XDK, especially item #3, to get a better understanding of why you shouldn't use the "browser technique" you're familiar with.

An alternative is to use a mapping tool that allows you to include the JavaScript directly in your app, rather than downloading it over the network each time your app starts. Several Intel XDK developers have reported very good luck with the open-source JavaScript library named LeafletJS that uses OpenStreet as it's map database source.

You can also search the Cordova Plugin Database for Cordova plugins that implement mapping features, in some cases using native SDKs and libraries.

How do I fix "Cannot find the Intel XDK. Make sure your device and intel XDK are on the same wireless network." error messages?

You can either disable your firewall or allow access through the firewall for the Intel XDK. To allow access through the Windows firewall goto the Windows Control Panel and search for the Firewall (Control Panel > System and Security > Windows Firewall > Allowed Apps) and enable Node Webkit (nw or nw.exe) through the firewall

See the image below (this image is from a Windows 8.1 system).

Google Services needs my SHA1 fingerprint. Where do I get my app's SHA fingerprint?

Your app's SHA fingerprint is part of your build signing certificate. Specifically, it is part of the signing certificate that you used to build your app. The Intel XDK provides a way to download your build certificates directly from within the Intel XDK application (see the Intel XDK documentation for details on how to manage your build certificates). Once you have downloaded your build certificate you can use these instructions provided by Google, to extract the fingerprint, or simply search the Internet for "extract fingerprint from android build certificate" to find many articles detailing this process.

Why am I unable to test or build or connect to the old build server with Intel XDK version 2893?

This is an Important Note Regarding the use of Intel XDK Versions 2893 and Older!!

As of June 13, 2016, versions of the Intel XDK released prior to March 2016 (2893 and older) can no longer use the Build tab, the Test tab or Intel App Preview; and can no longer create custom debug modules for use with the Debug and Profile tabs. This change was necessary to improve the security and performance of our Intel XDK cloud-based build system. If you are using version 2893 or older, of the Intel XDK, you must upgrade to version 3088 or greater to continue to develop, debug and build Intel XDK Cordova apps.

The error message you see below, "NOTICE: Internet Connection and Login Required," when trying to use the Build tab is due to the fact that the cloud-based component that was used by those older versions of the Intel XDK work has been retired and is no longer present. The error message appears to be misleading, but is the easiest way to identify this condition. 

How do I run the Intel XDK on Fedora Linux?

See the instructions below, copied from this forum post:

$ sudo find xdk/install/dir -name libudev.so.0
$ cd dir/found/above
$ sudo rm libudev.so.0
$ sudo ln -s /lib64/libudev.so.1 libudev.so.0

Note the "xdk/install/dir" is the name of the directory where you installed the Intel XDK. This might be "/opt/intel/xdk" or "~/intel/xdk" or something similar. Since the Linux install is flexible regarding the precise installation location you may have to search to find it on your system.

Once you find that libudev.so file in the Intel XDK install directory you must "cd" to that directory to finish the operations as written above.

Additional instructions have been provided in the related forum thread; please see that thread for the latest information regarding hints on how to make the Intel XDK run on a Fedora Linux system.

The Intel XDK generates a path error for my launch icons and splash screen files.

If you have an older project (created prior to August of 2016 using a version of the Intel XDK older than 3491) you may be seeing a build error indicating that some icon and/or splash screen image files cannot be found. This is likely due to the fact that some of your icon and/or splash screen image files are located within your source folder (typically named "www") rather than in the new package-assets folder. For example, inspecting one of the auto-generated intelxdk.config.*.xml files you might find something like the following:

<icon platform="windows" src="images/launchIcon_24.png" width="24" height="24"/><icon platform="windows" src="images/launchIcon_434x210.png" width="434" height="210"/><icon platform="windows" src="images/launchIcon_744x360.png" width="744" height="360"/><icon platform="windows" src="package-assets/ic_launch_50.png" width="50" height="50"/><icon platform="windows" src="package-assets/ic_launch_150.png" width="150" height="150"/><icon platform="windows" src="package-assets/ic_launch_44.png" width="44" height="44"/>

where the first three images are not being found by the build system because they are located in the "www" folder and the last three are being found, because they are located in the "package-assets" folder.

This problem usually comes about because the UI does not include the appropriate "slots" to hold those images. This results in some "dead" icon or splash screen images inside the <project-name>.xdk file which need to be removed. To fix this, make a backup copy of your <project-name>.xdk file and then, using a CODE or TEXT editor (e.g., Notepad++ or Brackets or Sublime Text or vi, etc.), edit your <project-name>.xdk file in the root of your project folder.

Inside of your <project-name>.xdk file you will find entries that look like this:

"icons_": [
  {"relPath": "images/launchIcon_24.png","width": 24,"height": 24
  },
  {"relPath": "images/launchIcon_434x210.png","width": 434,"height": 210
  },
  {"relPath": "images/launchIcon_744x360.png","width": 744,"height": 360
  },

Find all the entries that are pointing to the problem files and remove those problem entries from your <project-name>.xdk file. Obviously, you need to do this when the XDK is closed and only after you have made a backup copy of your <project-name>.xdk file, just in case you end up with a missing comma. The <project-name>.xdk file is a JSON file and needs to be in proper JSON format after you make changes or it will not be read properly by the XDK when you open it.

Then move your problem icons and splash screen images to the package-assets folder and reference them from there. Use this technique (below) to add additional icons by using the intelxdk.config.additions.xml file.

<!-- alternate way to add icons to Cordova builds, rather than using XDK GUI --><!-- especially for adding icon resolutions that are not covered by the XDK GUI --><!-- Android icons and splash screens --><platform name="android"><icon src="package-assets/android/icon-ldpi.png" density="ldpi" width="36" height="36" /><icon src="package-assets/android/icon-mdpi.png" density="mdpi" width="48" height="48" /><icon src="package-assets/android/icon-hdpi.png" density="hdpi" width="72" height="72" /><icon src="package-assets/android/icon-xhdpi.png" density="xhdpi" width="96" height="96" /><icon src="package-assets/android/icon-xxhdpi.png" density="xxhdpi" width="144" height="144" /><icon src="package-assets/android/icon-xxxhdpi.png" density="xxxhdpi" width="192" height="192" /><splash src="package-assets/android/splash-320x426.9.png" density="ldpi" orientation="portrait" /><splash src="package-assets/android/splash-320x470.9.png" density="mdpi" orientation="portrait" /><splash src="package-assets/android/splash-480x640.9.png" density="hdpi" orientation="portrait" /><splash src="package-assets/android/splash-720x960.9.png" density="xhdpi" orientation="portrait" /></platform>

Upgrading to the latest version of the Intel XDK results in a build error with existing projects.

Some users have reported that by creating a new project, adding their plugins to that new project and then copying the www folder from the old project to the new project they are able to resolve this issue. Obviously, you also need to update your Build Settings in the new project to match those from the old project.

Back to FAQs Main