Posts

Showing posts from 2013

Unix cheat sheet

Inspite of years of working on Unix and Linux I still wasn't confident if I knew enough and hence read it in these holidays and I see that we can get our way through these amazing OS using the below commands:- The OS uses the Filesystem to store data on the harddisk. They are different types of file storage, NTFS, FAT and etc.  Commands to get your through! pwd ==> Prints the working directory you are in.  ls ==> just to list the files and directories, flags -l  => output in list format -a => show the hidden files -h => output is more human, that is easier to understand  cd ==> Change directory  find location -type f -name filename   ==> to find the file that you have been looking for. parted for partitioning ext4 mkfs ==> to create filesystems echo $PATH  ==> print the path env ==> to list the environment variables of  ps -efl  ==> to print the details of the process that are running ps $$ ==

Accessing the Google APIs using Oauth 2 in Python

Image
In its simplest form, Oauth is as in this document I had created earlier. Oauth is such a simple protocol and so much of fuss is being made about it, its so simple that you can understand it even by reading its Specification  documentation ! Below pic shows the flow +--------+ +---------------+ | |--(A)- Authorization Request ->| Resource | | | | Owner | | |<-(B)-- Authorization Grant ---| | | | +---------------+ | | | | +---------------+ | |--(C)-- Authorization Grant -->| Authorization | | Client | | Server | | |<-(D)----- Access Token -------| | | | +---------------+ | | | |

Agile Python Development and the Laundry list

We need method to the madness! There are apparently two, one is waterfall and the other agile When? Manifesto for Agile Software Development was produced in February, 2001. Why Agile? Is more pragmatic, as it knows that not everything can be documented at the beginning of a project. But it does put in place some rigorous  checks to accommodate these changes! So that any change can be accommodated, with good communication, iterative design and development.  Agile development can be applied in any project, both testing tools and build automation tend to be very language specific. Python too has these tools. Jargons again! Agile methodologies  XP - Extreme programming -- focuses almost exclusively on the developer and development techniques DSDM - Dynamic Systems Development Method -- focuses completely on processes there could be many more. Spirit of it from manifesto! Individuals and interactions over processes and tools Working software over compreh

Google APIs for python - don't feel unfortunate anymore if you have to work on these

Image
From the past 4 years I've been unfortunate enough to work on Google API's which are those for which even Google cannot search their documentation, may be because it doesn't exist, stackoverflow no matter how much it overflows cannot give us anything much on these. Below is the summary of all my experience to crack the Google APIs. I've spent days together to crack one thing at times and I wished I had known these tricks so that it wouldn't have taken me this long. So firstly try to read as much as possible from the getting started guide. This will take you to a world of links, go to those link and grab in what ever is possible (don't worry if you don't understand anything). [trade secret of software engineers is make the simple sound as complex as possible, and intimidate others!] But the world of python is different!  " Simple is better than complex." -- zen of python So the magical commands in python are :- dir() type() help() (k

Installing Chef client in less than a minute

Image
Open a terminal Enter command as in the screenshot below, you will use the command as below and on the right get an acknowledgement curl -L https://www.opscode.com/chef/install.sh | sudo bash now after this we can confirm that the chef has been installed by looking at version by using the below command.  chef-client -v and also the folder structure 

Git simplified and master it in 10 minutes!

OK, what is this Git thing! Just because the developers use it, doesn't mean its rocket science. Some are using it to work on family photo albums so it should be useful for all. Imagine if you were composing a document and wanted a far off friend's help on it, you would write the initial draft, then email the attachment to your friend, he would download, save a copy, edit it and then email it to you. Now suppose instead of this emailing overhead, if we could just have this document stored on your computer and your final edits pushed and stored on a web repository which your friend can download from and work on it locally and push the changes to the web repo until its perfect. Different versions of this document from the initial one to the final one is stored and available and could be called Versions, since the docs are available with everyone working on it and not central its  distributed and you and your friend have a set of rules to control the flow, combing it, dis

OAuth Simplified

Image
In my earlier post we saw what is SAML. Basically it works on the HTTP i.e. for webpages and etc.  But what if you wanted have SSO on an app, for example on a Mobile App? There are work arounds to implement SAML but the straight forward way of doing it would be to use OAuth, a recent boy in neighbourhood which unlike SAML which is older than 6 years, is designed considering the future, of native apps and mobile apps.  As usual the definition is  OAuth is an open standard for authorisation. OAuth provides a method for clients to access server resources on behalf of a resource owner (such as a different client or an end-user). It also provides a process for end-users to authorise third-party access to their server resources without sharing their credentials (typically, a username and password pair), using user-agent redirections. Please observe the definition, it says Authorisation and not Authentication! What it means is that if you have an app on your phone and would like i

SAML (Security Assertion Markup Language) -- Simplified

Image
As discussed earlier SAML provides the web SSO as in the earlier blog on SSO. By Definition SAML is- Security Assertion Markup Language is an XML-based open standard data format for exchanging authentication and authorization data between parties, in particular, between an identity provider and a service provider. Breaking it down. IdP is the Identity Provider -- someone who has a database containing usernames and passwords. For eg. Google apps. SP is service provider -- a service we would like to use. For eg. apps like soundcloud and etc which allow us to log into their service using the credentials stored in IdP. So basically, if you want to add a comment to this blog post, you would have to authenticate with Google apps over the Browser and one Google identifies you as the right user, will allow you to access this resource and leave a comment. This is quite simply SAML at work. The below video is amazing and a must watch to just understand it further. For somet

SSO--Single Sign On simplified

Image
Before we go there, lets understand what is meant by Authentication and Authorisation. Lets take an example, if you are watching a video on youtube, its available to public and hence needs not Authentication, but to read your mail on gmail you would need to authenticate, i.e. say who you are and the gmail server checks by comparing your username and password with what has stored, this process is called Authentication. Now suppose you want be able to make changes to a resource, the system checks your Authority to see if are allowed to do that this is authorisation, like the uploader of a video can delete or edit the video where was as we are authorised to only view the video.  by definition Single sign-on (SSO) is mechanism whereby a single action of user authentication and authorisation can permit a user to access all computers and systems where he has access permission, without the need to enter multiple passwords. Single sign-on reduces human error, a major component of system

Error: SMTPIN_ADDED_BROKEN@mx.google.com

Some times we see the below in the email header information. SMTPIN_ADDED_BROKEN@mx.google.com The cause of the "...SMTPIN_ADDED_BROKEN@mx.google.com" is because of the 'https://' in the original Message-ID, which is not allowed. There could be some dodo systems which might be doing this and they would need to be tweaked to now allow message-id (see my earlier posts on what it means) and to have https!

Leverage the power of Cloud with Python

Image
My well honoured Presentation at the Cloud Connect International Event in Renaissance Hotel, Mumabi in 2013 Leverage the power of the Cloud using Python from Venu Murthy It was am amazing moment when my proposal was shortlisted and I was invited to speak in the much famed Cloud Connect event in Mumbai, June 2013. Below are some pics, the audience were amazing and as one of my friends said, "wow, you are listed with the C level people already, here !"

Google Mail or Gmail threading or conversations demystified

Image
Many of us like the mail threading or in Gamil they are also called as conversations and for those who do not like it, Google does gives us the option to disable threading. To do so, go to the General tab of your Gmail Settings, and select the radio button next to 'Conversation view off'.  However for those of you who are interested in Gmail threading. Below is a threaded mail Gmail threading is based on these factors.  Subject must be same or similar (it can have a prefix of Re:, Fw:) Sender must be a part of the thread OR in- reply-to must be used. (see this link for the basics)  Time from when the original was sent and its reply. A typical header message looks as below:- Message-ID: <13dsaf693.25715.YahooMailNeo@web126206.mail.ne1.yahoo.com> Date: Mon, 23 Sep 2013 04:44:53 -0700 (PDT) From: Divakar Nadig <xyz@yahoo.com> Reply-To: Divakar Nadig <xyz@yahoo.com> Subject: please see this dear son..... To: * V e n u * <xyz@gmail.

Whats an email anyway? What are it's components?

Image
A some what weird question but when explored its very interesting and lets see how deep the rabbit hole goes! Electronic mail is commonly called as e-mail or eMail, i ts been around from 1973.  it’s a method of exchanging digital messages across the internet or other computer networks. After the Snowden era may be we will see a trend of peer to peer communication without a centralised server, the way it was in the beginning of time!  An Internet email message contains 3 components:- Message envelope Message header – contains control information, flow and etc. Message body In a client-server model, client is from where the message originate and  Email servers accept, forward, deliver and store messages.  Originally emails could carry text-only (7-bit ASCII) communications medium, which was then extended to carry multi-media content attachments, a process standardised in RFC 2045 through 2049. Collectively, these RFCs have come to be called Multipurpose

The best IDE for Python -- PyCharm goes opensourced - i.e. Free!!!

It gives me great joy that PyCharm has been opensourced, I've been using it for the past 2 years and am kind of addicted to it, though I do agree that I haven't exploited it enough. You can download it from this link .  Whats so great about it! Awesome Navigation Command + Click on a module to go to where the class is defined. Option + F7 gives a view of where all its used. Command + Option + B ==> to see where all its implemented. What I love the most is the way it generates functions for you. Type name of a method which has not yet been defined. and then Option + Enter and select Create Function Option + Command + V will instantiate an object Enjoy Python!

Simply put whats an API anyway?

Image
API in robotic terms is  application programming interface which ofcourse doesn't make any sense.  So to understand that in a simple way lets see what is a GUI, again in robotic terms its Graphic User Interface. i.e what ever action you want to take a button or something of the sort could be clicked. Like to close a browser you would take the mouse pointer to the x on the window.  However if it was the end of your day and you were closing 20 such windows of various apps on your laptop, you know the pain of it. However if you could just automate it!  Now imagine if there were 2000 such windows to be closed, you would want your computer do something about it but how will you tell your computer on what to do? This is were the APIs come into picture, they are a programatic way of doing manual things on computers.  Basically APIs are a replacement for GUI if you want to automate a lot of stuff.  Eg.  If you are Google apps admin with no LDAP sync and etc and ne

I am late and where was I

I wonder where I was all these days and why did I not blog. And as its human to give excuses, I would say that the companies I worked for never gave me the leisure to either read a blog or post one in peace. But I am going to force myself to share all that I've learnt so far, as this is the best way for me to give back to the internet in my humble what it has given me for more than a decade.