Pure Python vs NumPy vs TensorFlow Performance Comparison

This Post Was Originally Published on Real Python on May 7th, 2018 by Renato Candido.

Python has a design philosophy that stresses allowing programmers to express concepts readably and in fewer lines of code. This philosophy makes the language suitable for a diverse set of use cases: simple scripts for web, large web applications (like YouTube), scripting language for other platforms (like Blender and Autodesk’s Maya), and scientific applications in several areas, such as astronomy, meteorology, physics, and data science.

It is technically possible to implement scalar and matrix calculations using Python lists. However, this can be unwieldy, and performance is poor when compared to languages suited for numerical computation, such as MATLAB or Fortran, or even some general purpose languages, such as C or C++.

To circumvent this deficiency, several libraries have emerged that maintain Python’s ease of use while lending the ability to perform numerical calculations in an efficient manner. Two such libraries worth mentioning are NumPy (one of the pioneer libraries to bring efficient numerical computation to Python) and TensorFlow (a more recently rolled-out library focused more on deep learning algorithms).

  • NumPy provides support for large multidimensional arrays and matrices along with a collection of mathematical functions to operate on these elements. The project relies on well-known packages implemented in other languages (like Fortran) to perform efficient computations, bringing the user both the expressiveness of Python and a performance similar to MATLAB or Fortran.
  • TensorFlow is an open-source library for numerical computation originally developed by researchers and engineers working at the Google Brain team. The main focus of the library is to provide an easy-to-use API to implement practical machine learning algorithms and deploy them to run on CPUs, GPUs, or a cluster.

But how do these schemes compare? How much faster does the application run when implemented with NumPy instead of pure Python? What about TensorFlow? The purpose of this article is to begin to explore the improvements you can achieve by using these libraries.

To compare the performance of the three approaches, you’ll build a basic regression with native Python, NumPy, and TensorFlow.

Read More

Using Python Fabric to automate GNU/Linux server configuration tasks

Fabric is a Python library and command-line tool for automating tasks of application deployment or system administration via SSH. It provides tools for executing local and remote shell commands and for transferring files through SSH and SFTP, respectively. With these tools, it is possible to write application deployment or system administration scripts, which allows to perform these tasks by the execution of a single command.

Read More

Controlling a led over the Internet using Arduino and Ethernet Shield

Ethernet Shield enables us to connect Arduino to an ethernet network and build projects which send and receive information via local network and internet. The device, along with its control library, allows the design of network applications easily. There is no need to worry about the complicated signaling schemes in ethernet networks since all control and connection are made by library routines. After the connection is established, we may just read or write data through it, similarly to a conventional serial connection.

In the following, we show an example of a project of a web page that allows you to turn on or turn off a digital port of the Arduino via the local network or the internet. In a practical application, the project could be used to control a lamp, serving as a good “Hello World” application for home automation with Arduino. In this example, the idea is to focus on the design of the software required for the application so that we will consider the connection of led to the Arduino port and show one of the most expensive ways to light a led.

Read More

E-mail backup with NoPriv.py

Listening to the Linux Action Show podcast, I heard about a Python script to backup e-mails from IMAP accounts that downloads messages and attachments and offers everything on a nice local HTML page.

The script is called NoPriv.py and can be found on this page and on this Github repository. Given an IMAP e-mail account and a list of folders to be copied, it creates the HTML files structure to access the messages as shown in this demo page. The backup can be made incrementally, transfering only the new messages each time the script is run.

Read More

The right to read

“The right to read” is an article by Richard Stallman which was published formerly on an issue of Communications of the ACM and later in the book “Free Software, Free Society: Selected Essays of Richard M. Stallman“. In the article, he discusses about the possible changes regarding the reader’s freedom due to adoption of e-books instead of traditional books. Good reading!

The right to read

philosophical-gnu-sm[This article appeared in the February 1997 issue of Communications of the ACM (Volume 40, Number 2)].

From The Road To Tycho, a collection of articles about the antecedents of the Lunarian Revolution, published in Luna City in 2096.

For Dan Halbert, the road to Tycho began in college—when Lissa Lenz asked to borrow his computer. Hers had broken down, and unless she could borrow another, she would fail her midterm project. There was no one she dared ask, except Dan.

Read More

More courses on the web

It’s been a while since the emergence of university courses on the web. Currently, almost all major universities maintain projects regarding on line courses such as Open Course Ware provided by MIT and Open Yale Courses, by Yale University.

In these projects, at first, only written material such as student’s notes and exercises were provided. However, with the arise of video distribution through the web, it didn’t take long before cameras were used to record the classes and make the videos available on the web.

Read More

Hello World with Arduino

In 2005, I came across a project called Arduino, which drew my attention since it was a project of open source hardware. At the same year, I tried to assemble one but I ended leaving the project aside due to the college activities. Today, I regret I haven’t dedicated more some time to this project in the past, given the way it grew.

For those who don’t know, Arduino is a project composed by hardware and software for the development of applications which interact with the real world using sensors and actuators. It was conceived to permit the creation of projects by people who don’t have much knowledge on hardware or electronics such as hobbyists, artists and designers.

Read More

Python for scientific applications – the very basics

Lately, I’ve been studying a little about Python and the more I learn the more I become interested about this language. It was the first time I had contact with a language that uses dynamic typing and since the first time I used it, I had the impression it could be a good replace for Matlab. I like Matlab, but the fact of being proprietary and not being a general-purpose language makes Python a better solution for me.

Read More

Organizing with Emacs Org-Mode

Another interesting Emacs mode is Org-Mode. It permits to make annotations and manage to-do lists and projects using text files with some special marks.

The marks as interpreted by Emacs, so that it separates the various parts of the text, which facilitates obtaining information. It is possible to organize an agenda with appointments set up on specific dates and also tasks without due dates. With every agenda item, it is possible to associate tags, which can be used to filter search results. These features make Org-Mode a good resource for those who use GTD. For those who don’t know GTD, it is worth to take a look. It was proposed by David Allen and is described in the book “Getting things done”. A brief description of the method can be seen on this presentation which was given at Google and on this reference card.

Read More