Monthly Archives: October 2017

DATA MINING WITH PYTHON

Data Mining With Python

Data mining is the extraction of implicit, formerly unknown, and potentially useful information from data. It is applied in a wide range of domains and its practices have become fundamental for several applications.

This article is about the tools used in real Data Mining for finding and describing structural patterns in data using Python. In recent years, Python has been used for the development of data-centric.

DATA IMPORTING AND VISUALIZATION

The very first step of a data analysis consists of obtaining the data and loading the data into the user’s work environment. User can easily download data using the following Python capability:

import urllib2

url = ‘http://aima.cs.devopspython.edu/data/iris.csv

u = urllib2.urlopen(url)

localFile = open(‘iris.csv”, ‘w’)

localFile.write(u.read())

localFile.close()

In the above snippet user has used the library urllib2 to access a file on the website and saved it to the disk using the methods of the File object provided by the standard library. The file contains the iris dataset, which is a multivariate dataset that consists of 50 samples from each of three species of Iris flowers. Each sample has four features that is the length and the width of sepal and petal, in centimetres.

The dataset is stored in the CSV format. It is appropriate to parse the CSV file and to store the informa tion that it contains using a more suitable data structure. The dataset has 5 rows, the first 4 rows contain the values of the features while the last row signifies the class of the samples. The CSV can be easily parsed using the function genfromtxt of the numpy library:

from numpy import genfromtxt, zeros

# read the first 4 columns

data = genfromtxt(‘iris.csv’,delimiter=’,’,usecols=(0,1,2,3))

# read the fifth column

target = genfromtxt(‘iris.csv’,delimiter=’,’,usecols=(4),dtype=str)

In the above example user has created a matrix with the features and a vector that contains the classes. The user can also confirm the size of the dataset looking at the shape of the data structures loaded:

print data.shape

(150, 4)

print target.shape

(150,)

print set(target) # build a collection of unique elements

set([‘setosa’, ‘versicolor’, ‘virginica’])

An important task when working with a new data is to understand what information the data contains and how it is structured. Visualization helps the user to explore the information graphically in such a way to gain understanding and insight into the data.

CLASSIFICATION

Classification is a data mining function that allocates samples in a dataset to target classes. The models that implement this function are called classifiers. There are two basic steps for using a classifiers: training and classification. The library sklearn contains the implementation of many models for classification.

t = zeros(len(target))

t[target == ‘setosa’] = 1

t[target == ‘versicolor’] = 2

t[target == ‘virginica’] = 3

The classification can be done with the predict method and it is easy to test it with one of the sample:

print classifier.predict(data[0])

[ 1.]

print t[0]

1

In this case the predicted class is equal to the correct one (setosa), but it is important to assess the classifier on a wider range of samples and to test it with data not used in the training process.

CLUSTERING

We do not have labels attached to the data that tell us the class of the samples. The user has to analyse the data in order to group them on the basis of a similar criteria where groups are sets of similar samples. This kind of analysis is called unsupervised data analysis. One of the most famous clustering tools is the k-means algorithm, which can be run as follows:

from sklearn.cluster import KMeans

kmeans = KMeans(k=3, init=’random’) # initialization

kmeans.fit(data) # actual execution

The snippet above runs the algorithm and groups the data in 3 clusters (as specified by the parameter k). Now the user can use the model to assign each sample to one of the clusters:

c = kmeans.predict(data)

And the user can evaluate the results of clustering, comparing it with the labels that they already have using the completeness and the homogeneity of the score:

from sklearn.metrics import completeness_score, homogeneity_score

print completeness_score(t,c)

0.7649861514489815

print homogeneity_score(t,c)

0.7514854021988338

The wholeness of the score approaches 1 when most of the data points that are members of a given class are elements of the same cluster while the homogeneity score approaches 1 when all the clusters contain almost only data points that are member of a single class.

The user can also visualize the result of the clustering and compare the assignments with the real labels visually:

figure()

subplot(211) # top figure with the real classes

plot(data[t==1,0],data[t==1,2],’bo’)

plot(data[t==2,0],data[t==2,2],’ro’)

plot(data[t==3,0],data[t==3,2],’go’)

subplot(212) # bottom figure with classes assigned automatically

plot(data[c==1,0],data[tt==1,2],’bo’,alpha=.7)

plot(data[c==2,0],data[tt==2,2],’go’,alpha=.7)

plot(data[c==0,0],data[tt==0,2],’mo’,alpha=.7)

show()

RISKS & MYTHS OF CLOUD COMPUTING

Risks & Myths of cloud computing

Despite the rapid adoption of cloud computing and its positive impact on business, there are still some myths and risks, concerns and misinformation around deploying and running applications in the cloud.

In this article we will be going to discuss the myths and the risks and provide the correct information and make you take an informed decision for your business or organization.

MYTH: EVERYTHING WORKS BETTER IN THE CLOUD

Cloud computing has its place in every business and organization, from multi-national corporations and financial institutions to online businesses and start-ups. The common profits of the cloud are faster time to market and streamlined processes and flexible infrastructure costs which are difficult to ignore. But just because the cloud has its place in every business, does not mean it is right for every task.

The public cloud might be best for some actions, private cloud for others, and dedicated hosting for legacy applications. Getting the right overall solution often requires a combination public, private, and dedicated infrastructure.

RISK: SECURITY RISKS AT THE VENDOR

When a cloud service vendor provides a critical service for your business and stores critical data – such as customer payment data and the mailing lists then you are placing the life of your business in the vendor’s hands.

Many small businesses know almost nothing about the people and technology behind the cloud services they use.

When the person is depending on a cloud service for their business-critical task, then they put the trust of their business into the hands of other people and the quality of their work.

The reputation is no longer depends on the integrity of only their business – now it also depends on the integrity of the vendor’s business. And that’s a cloud computing risk.

MYTH: DATA IS NOT AS SECURE IN THE CLOUD

A security breach could bring down your site and may cause the users to lose their valuable revenue. Therefore, it is not surprising that security concerns are one of the largest hurdles for many businesses considering the cloud. However, security is also one of the easiest cloud myths to debunk.

In reality, security risks in the cloud are the same as those faced by traditional IT solutions, but with one main difference: When operating in the cloud, security no longer rests on your shoulders alone. Instead, security is a shared responsibility with the cloud hosting provider.

RISK: RISKS RELATED TO LACK OF CONTROL

When the user host and maintain a service on a local network, then the user has complete control over the features he/she choose to use. However, when the user uses a cloud service provider, the vendor will have the control. The user has no guarantee that the features using today will be provided for the same price tomorrow or not. The vendor can double its price, and if the clients are depending on that service, then the user might be forced to pay. Also, who controls access to the user’s data in a cloud service? What happens if the user is not able to make payment?

If the user gets behind on their bill, then the user may be surprised to find their data is held hostage by the vendor. The user cannot access the service and export the data until he/she pay up.

MYTH: IT IS ALWAYS CHEAPER TO RUN IN THE CLOUD

It is not always cheaper to run in the cloud, but it can often be more cost effective. If you need all your servers running 24x7x365, it is likely you can get the same compute power for less cost using a dedicated server.

Cloud works best for variable demands and workloads, where you have higher demand at certain times and lower demand at others. The cloud allows to switch servers off during periods of lower demand to improve cost efficiency by more closely matching the cost pattern to the user’s revenue/demand pattern.

RISK: AVAILABILITY RISKS

No service can guarantee 100% uptime. When you rely on a cloud service for a business-critical task, then you are putting the feasibility of the business in the hands of two services: the cloud vendor and ISP.

If the user’s internet access goes down, then it will also take your vendor’s cloud service with it.

MYTH: CLOUD TECHNOLOGY IS STILL IN ITS INFANCY

A latest ISACA study revealed that cloud computing is fast approaching, within the next few years, you can expect to see constant innovation at an ever-increasing pace. Constant refinement will help to ensure that cloud computing meets the needs of every size and type of business. Those who harness the cloud now will be the first to reap its long-term rewards.

HOW TO IDENTIFY THE MOBILE SCAMS?

How to identify the mobile scams?

In today’s world, much of Internet’s traffic is from mobile devices. This makes mobile users just as much of a target as computer users. Mobile scams and malware are on the rise, and scammers are budding to get smarter at tricking the user into giving up the user’s data. Explore the different types of mobile scams below and the best ways to identify them and keep your data safe.

SOCIAL MEDIA SCAM

If you are on Twitter, Facebook, and other social media sites you know how annoying it is to be followed by a confused name with zero followers, then receive a tweet with little more than a link. Facebook is full of fake profiles, some of which have acquitted intentions, but most are there to provide fake Likes on demand – or to spam genuine users with phishing links. However, never click on a link sent by an unfamiliar person. Even if it seems harmless enough at the time, you could end up either giving your valuable personal details to a criminal or in worse scenario have malware inserted onto your mobile.

SMISHING

Same old-fashioned methods are applied over SMS. User’s will get a message from unknown numbers influencing the user to reply to a number or to click on a link and open it in the user’s phone browser. This might likely insert malware onto the user’s phone or alerts the scammers to the fact that the number is active and worth targeting again.

These scams can present themselves in many forms. For instance, misleading offers such as “free” ringtones, lotteries offer, and much more. There are few ways scammers can get the user with ringtone scams. Sometimes those “free” ringtones are an actual subscription service, and if the user’s respond he/she may unknowingly have signed up for the service. Other times, they come embedded with malware.

SMS phishing or Smishing will use some scary tactic to get a quick reply without much time to think about the action. When in the form of a notice from a bank or financial institution, the phisher requests immediate action or the user’s account will be closed. If the user feels that his/her accounts are in danger, check for the company’s customer service phone number online and call them to verify the text.

CLONING

Scanners will read the user’s cell phone identity, including the number and its unique serial number, then they program another phone with the same details and make calls at the user’s expense.

VOTING WITH PHONE

People will receive text messages or recorded messages offering them the chance to cast their vote through phone, simply by pressing a key for each of the candidates.

This is one trick targeting the voters of one political persuasion or another, to stop the voters from actually casting their vote for real.

SINGLE RING SCAMS

Scammers will call once and end up the call, with the objective of tempting the user to call back to that number. These numbers can be high toll lines, and by calling them the user might end up paying a premium rate for the phone call.

TOP 5 DEVOPS PREDICTIONS FOR 2017

Top 5 DevOps predictions for 2017

2016 was good for DevOps: We saw more enterprise adoption of containers and more organizations throwing their hats into the container ring. That doesn’t mean that the tools surrounding DevOps are mature. According to experts, 2016 set the stage for security enhancements, containerization, and consolidation.

The next phase of DevOps will focus more on security. In 2017, building security practices as code will be part of application development, rather than applying it post-facto. This will lead to DevOps going beyond Dev, QA, and Ops. So, explore this article and the top five DevOps predictions for 2017.

TOP 5 DEVOPS

Considering technology on front, we will see increase in popularity of containerization solutions Docker because of its ability to provide constant environment from development to production. Next year it will be more popular with non-production environments, and as it develops it will see similar popularity for production environments. One of the key reasons for this popularity is its portability across multi-cloud platforms.

EFFORTS TO ADDRESS ENTERPRISE CONCERNS WILL INCREASE

With an increased experience of implementing next gen platforms and automatically generating containers, there will be a greater focus on enterprise concerns, such as access controls, audit trails, and network technologies that can implement “virtual firewalls” at the level of the orchestration tier.

DevOps will extend into pay-as-you-go

We will see more cloud implementations of DevOps to meet the needs of an on-demand model. Technology solutions which orchestrate across cloud providers will only accelerate that adoption by abolishing the risk of cloud provider locking. Customers can easily switch over to a low-cost provider and can profit from the elastic nature of the cloud pricing model.

MORE AUTOMATED CODE

In 2016, organizations began introducing tools to decrease the tedium of finding a line of error code in applications, and 2017 will see more automation for developers. The automation will revolve around code testing, gathering and formatting data, reporting, and notifications.

“Coding through automation and machine learning will be more dominant than previous years, but now it is possible due to new hardware and techniques” such as GPUs and parallel computing.

CLOUD MIGRATION AND ITS ADVANTAGES

Cloud Migration and its advantages

Over the past few years, cloud computing has taken the tech world by storm. This computing model is now making it easier to deliver computing resources to businesses worldwide, and at their current costs. The companies have reduced exponentially owing to automation, and the internet-powered cloud migration strategy is already helping countless businesses worldwide to make smooth transitions. We know that the fast adoption of cloud technology is not really far away. Today, many companies have started their journey towards an integrated cloud environment by migrating their existing systems, applications, services, and data to the cloud. So, explore this article and know more about cloud migration and its advantages.

CLOUD MIGRATION

Cloud migration involves the process of moving relevant business data, applications, and other important elements of a company from its desktops and servers to the cloud. This process can sometimes also involve the moving of data between different cloud environments provided by different service providers based upon their area of expertise.

Cloud migration makes the cloud computing possible, wherein the cloud performs all the computing that was performed earlier by mobile devices, laptops, or desktops. The benefits of cloud computing are many, as most of the background processing is now presented at large, secure data centres thereby saving valuable organizations resources. At the same time, cloud migration challenges are still an important reality that many organizations wake up to unless they partner with an experienced service provider.

ADVANTAGES OF CLOUD MIGRATION
IMPROVED SECURITY FEATURES

Most of the cloud providers take care of the tougher security problems, such as keeping unwanted traffic outside a specific scope from accessing the machines on which the user’s data and apps reside and ensuring automatic security updates which are applied to their systems to keep from being vulnerable to the latest known security threats.

Keep in mind, that the user still need to have security policies in place on their end, such as keeping mobile devices secure, making sure employees do not reveal passwords or other sensitive information to unauthorized parties, and much more.

LESS SUBSTRUCTURE COMPLEXITY

Cloud systems always tend to peel away the complexity of the substructure that underlies the architecture being used to provision new machines and make them all work together to provide the needed services. Instead, the user can fill out some information on what is needed and launch the necessary services. This can save quite a bit of time, as those particular complexities are no longer a part of the process.

AUTOMATIC BACKUP AND LOGGING OF KEY METRICS

Monitoring, backup, and logging services are extremely important, especially if the user need to perform disaster recovery from an outage and see where things went wrong. The backups will allow the user to get things up and running again, and the logs may provide some critical information to help the user to find out what caused the issue in the primary place.

FLEXIBLE

Flexibility is something which comes to our mind when we speak about cloud migration. It is best suited for organizations with changing bandwidth demands as the services will be customized accordingly. If the needs increase, it is easy to scale up the cloud capacity.

IMPROVED COST MANAGEMENT

Some cloud providers also provide auto scaling, which allows the user to provision more services when needed and turning them off when they are not needed. This highly responsive technique can help even more with price savings, as the user only needed to be charged for the additional systems instead of keeping additional machines up and running all the time to deal with peak load. In this way, the services can automatically respond with the number of resources needed at any given time, preventing both downtime and unnecessary expenses.

WHAT KIND OF INFORMATION DO INTERNET OF THINGS (IOT) OBJECTS COMMUNICATE?

What kind of information do Internet of Things (IoT) objects communicate?

The answer depends on the nature of the object, and it can be simple or complex. For example, a smart thermometer might have only one sensor, used to communicate ambient temperature to a remote weather-monitoring center. A wireless medical device might, in contrast, use various sensors to communicate a persons body temperature, pulse, blood pressure, and other variables to a medical service provider via a computer or mobile phone.
 
 
 
 
 
 
 

WHAT IS THE DIFFERENCE BETWEEN IOT AND M2M?

What is the difference between IoT and M2M?

The difference between M2M and IoT are as follows.

M2M is basically communication between a machine or a device with a remote computer.

There are three processes involved in the M2M as mentioned below:

  • It connects the device to the cloud.
  • Manage the device.
  • Collects the machine and/or sensor data.

IoT goes beyond the M2M periphery and it basically represents things connecting with the systems, people, and other things.

M2M can be taken as a integral to the IoT. IoT is bigger than M2M.

WHAT IS GARBAGE COLLECTION AND DOES PYTHON HAVE IT?

What is garbage collection and does python have it?

Garbage collection is the systematic recovery of pooled computer storage that is being used by a program when that program no longer needs the storage which frees the storage for use by other programs.

Python also have an inbuilt garbage collector, which recycles all the unused memory and frees the memory and makes it available to the heap space.

 
 
 
 
 
 
 
 
 

EXPLAIN MULTI-THREADING IN PYTHON?

Explain multi-threading in Python?

By default Python doesn’t allow multi-threading to use multi-threding we have to use different multi-threading packages.

Multi-threading can be outsourced to the operating system (by doing multi-processing), some external application that calls your Python code, or some code that your Python code calls.