The Web!

As it turns out, there is this one overlap of interests between my needs and my University studies this semester - Finally!

This semester, I have the subject " Internet and Web Engineering" which aligns quite well with one of my goals regarding the JavaScript, CSS and HTML5 exploration which I need to accomplish many goals like - Understanding the Browsing experience, the How of Internet and most importantly to improve these blogs using the knowledge that I gain through this experience. Renovating The Blog has already been on my ToDo list for a while now.

I haven't made any progress on any of the projects which I have outlined in a previous post - been quite busy with the University project. I have learned quite a bit about the data formats and made good acquaintance with the wonderful Bokeh library, realizing in the process the areas which I need to work upon like improving my understanding of Web Pages, especially JavaScript.

With in the University project I have explored HDF5 data storage format and of course, for the time being I have switched over completely to my Desktop ( Ubuntu ) environment.

Right now, I am working on the various API ( Application Protocol Interface ) which is basically related to the Facebook " Like" button you see all over the Internet! But in my case, I need to use the Twitter API and then use the NLTK to analyse the overall emotional tone of the Tweet.

Apart from this, been doing a bit of exploration in the world of Python and I found a few interesting libraries which are related to Music - Python is definitely my Mother Tongue;P

Will summarize about these Music Libraries in the next post!

You can run but you can't hide - JavaScript !

Yup! JavaScript is one thing that keeps coming up everywhere be it GeoGebra, Adobe Acrobat or The Internet. The "public opinion" is much divided whether the language really is worthy of having the status that is accorded to it.

But putting the multiple criticism aside, when I  think of what JavaScript would enable me to achieve is enough for me to go through with it anyway.

Things like,

** Being able to control the behavior of my Web Browser, especially in light of the countless advertisements and pop-ups I am bombarded by every time I wish to surf the Internet. Seriously people, Wayy too many useless ads on sites these days!

** The ability to able to code with absolutely no Environment-setup is a definite plus point for me. I would love to be able to Own the Web-browser and the be able to solve problems, experiment with various approaches even when I am on a computer as a guest or say, on a Computer which is not quite up to date. Another reason in Favor of JavaScript!

** It's a good skill to have, I wish to be as independent in future as possible so working on putting up a web-site et cetera is easy as well as a desirable skill. Plus, I get to improve my own blogs through this experience.
                     
                              Well, I think this is another reminder of the promise that I made to you guys previously. About giving this blog a good upgrade, a bit of personality.

** Being able to experiment with zillions of JavaScript libraries like D3.js, Node.js et cetera for visualizations and then to port those ideas over to Anaconda Scripts is another way to improve my skills in Python.

I truly believe that understanding the Internet is only going to help my cause in Future. I need to understand how the Internet works - It's gonna simplify my life to no end. There is no escaping the Internet, why not own it;P

Though I am not entirely unaware of the fact that Internet is an ocean of Technologies so there would always be so so many things that I don't know but with JavaScript literacy I would be able to learn new things in a more effective way for my purpose.

Again, the PURPOSE is to be able to do Mathematics and Physics with Computers. Not once have my conviction wavered and I assure you all this experimentation is gonna end up improving my computer literacy manifolds. I don't have time to go through all the exercises in a Mathematics books, the same is true with the Physics books. My solution - grasp the concepts and always aim for tough problems which are an intellectual challenge and help us consolidate the knowledge that we have earned through the theory by practical application. Fast and Elegant solutions - all done at a lightning fast speed by my buddy Computer.

As Richard Feynman said :-

"What I cannot create - I do not understand !"

Implication being - " Know how to solve every problem that has been solved". 

In this age, in our times , it's not about manual labor - it's about the sheer Creativity that we can unleash.

Oh!

Good News, I have switched over to the LINUX computer, at least for the length of this project. I think, I am totally gonna have a LINUX Desktop in future, with a Mac book Pro as my Working environment. That's the environment that suits my believes and my working style.

I am soon gonna start posting these posts from within Emacs. That's one piece of software that surprises me continuously. Totally in love with the Emacs and Elisp. Looking forward to getting back to the

P.S. I notice that being able to change things, programatically does make one, more of a control freak. All the same no, "With Great Powers, comes Great Responsibilities";P

Approaching Complex Problems - Break 'em Down!

As you guys know, I am attempting to make a full-fledged Beast of a Program right now - which sort of provides me with the Opportunity to put into practice the various parts as a coherent whole.

I have made progress along the lines of Tools that I would be needing as well as the Resources ( web-sites for the statistics) that I would be needing to build the final visualizations upon. As I have, mentioned a few times in my previous posts regarding Data Artists, I think that when we are able to See what's happening in that obscure world of numbers - we are more into our Zone. I have come to believe that if there is one thing our senses excel at ( unless deliberately trained ) it's the sense of Sight. So, I am doing my best to try out my hands with Visualizations.

Another thing that I have learned from handling this complex a problem is that it is best to chunk 'em up and then approach the little problems, solving them as we move towards other parts of the problems and then, once we have the working prototype of a solution - Combine them into an Organism.

Basically, the end result of all the effort that I put into this project is a bunch of visualization on the HTML using Bokeh, which you can have a taste of in these Jupyter Notebooks.

I think, it's as true for Problem-Solving as excelling at any physical activity that involves learning a new skill. Ever seen someone who just "gets it naturally" once they have seen it for a while?

Call it Beginners Luck or Clear Mindedness but I am starting to believe that the Master sees a number of parts done right, pays attention to the minute details whereas the Student can only see the rhythmic whole. There are more levels of Beauty for a Master.

I won't say I am a Master yet - but that's not to say I won't be one;P

The Windows of Opportunity - Act 2

The thing is with my current University project, I would like to use the tools that I have been working hard for the past couple of months and I am gonna use the experience to improve my skills and confidence in Programming ( as well as in my Tool-Kit ) by tackling tough problems.

As of now, the project seems to be heading towards the direction of Text Processing- Visualization - Data Fetching and Of course Statistics to analyse all that Information and ultimately to find a meaningful way to visualize that data.

It's for that end part of the project that I seek to go through it all - Visualization. Remember my post about Data Artists and Visualization? Yeah, I get to experiment with it all on my own.

However, I am aware of the fact that the time I am gonna invest in this project would have to come at the price of other projects that I wish to complete this year. But, all in all - I think that this is a brilliant opportunity to push my boundaries.

What I seek to learn from the project?

I think, that I won't have the opportunity to work directly with SageMaths ( Just had a Major release btw, check it out ASAP ) but I would definitely be writing a lot more Anaconda Scripts.

I don't exactly know what libraries I would be making use of, for the project but so far these are the ones which seem highly relevant -

* Pandas
* NumPy
* Requests
* Beautiful Soup
* NLTK ( Natural Language Tool Kit)
* Lxml
* Bokeh
* MatplotLib

Apart from these, I have no idea about the what other stuff I might have to look into. 

I doubt it very much that I would have the opportunity to use Emacs or even other programming languages in any way but I suppose that I am gonna learn to really handle Real-World programming problems and get to Be Creative with the way I choose to Visualize them.

Through the project, I aim to learn Text Processing i.e. how to use the standard Text Processing tools that Python Library provides and Regular Expressions, along with NLTK which would be immensely helpful in working with Human Languages - the project would consist of analysing sentiments from the text that has been downloaded using the various Web-site data clients. No idea, how to do that as of now btw;P Will learn, though.

I get to learn how to connect to web-sites, something which I have been exploring recently and how can I change that data to a form which would be accepted by the other libraries which are involved in the project. Thus ingenious solutions would be necessary, good challenge.

The total Lines Of Code ( LOCs) might be more than any other program that I have ever worked on. So, I might practice multiple file approach to build huge programs. Needless to say that this is gonna be a great field-testing for my experience with Git for versioning various levels of progress.

Again, this is not directly related to Maths or SageMaths but I do have a couple of Ideas at the back of my head which might be quite useful if I master the art of Text processing. Of course, once I do it in Python then it can also be done in Emacs and then again in LINUX once I switch to the Unix systems.

So, NLTK + Text Processing is the thing for me right now. My main focus for now.

Besides, I intend to present my work via interactive graphs - something along the lines of the Videos below. Bokeh is the alternative to D3.js which is pretty much the most famous visualization library out there - the plus point with Bokeh for me being I get to use Python (not Javascript, which I have no experience with) and I intend to explore the entire Computer world via Python first and only later move on to specific tools as the necessity dictates. So, with Bokeh, being a web-browser based technology I am gonna stay in touch with the Internet and how web-pages are structured.

Would take a lot of work for me to get them done but I aim to work hard and get it all done in a month!

One thing, that I am sure of now - I won't be using Chrome any more, it's so glitchy. Atleast on my WIndow7+ i5 laptop. It might be different on the Linux and on my Desktop but there is another factor that's moving me closer to Mozilla Firefox - it's Rust.

Nope, not the one from True Detective but rather it's a Programming Language which is getting quite a lot of attention these days. I find it to be better as compared to Google's Go language so that's that for me.  Anyway, I would like to leave the rest in the Expert Hands of Rust Cohle - Have Fun!!

The Windows of Opportunity - Act 1

Hey, I am super exited about using these Spells that I have been working on for projects! First part of this post is gonna summarize the history of the University project so far.

Actually, the thing is it's compulsory for the 4th ( and final year) students to work on a project in their final year - it's named the "Major" project. But there was already a lot of practice during the "Minor" in the 3rd year. So during the minors ( 1 per semester ), I gradually moved the project towards the languages, of course, human languages;P

The thing is there is so much that can be done with in this context. From a programming point of view this involves heavy text processing and, among many other alternatives, some heavy statistical analysis.

As my skill in programming weren't very brushed at the time, I chose to go with the Closed-Source Computer Aided Mathematics system, Mathematica. I absolutely loved Mathematica, especially the vision of it's founder and CEO who himself is pretty awe-inspiring. The other big name in CAM's is of MATLAB but the Mathematica people seem to be more active in arousing interest in the people regarding the endless possibilities of creativity in Mathematics, once the drudgery and fear of "complex" things is removed. 

I absolutely recommend that you have a look at the following links and would really encourage you to look up more videos by Stephen Wolfram and even to find out more about his background. One person, I find who believes in what he says.

Teaching Kids Real Maths

Computer Based Maths

A Theory of Everything

Besides, the Mathematica language is being distributed free with Raspberry Pi systems. So, apart from the official trial period you can just purchase a Raspberry Pi and have a good go at it.

And, I don't really believe in "Camps" as in Mathematica is "Good" or "Bad" for such and such reason. But I would definitely look it up thoroughly and decide for my own whether it suits my purpose. 

If money is not an issue and you have only to do preliminary research with Computers and not willing to really understand how the internals work, then Mathematica is definitely worth looking into.

Hey, What about SageMaths?

Obviously, I LOVE SageMaths and I have no doubt in my mind that you are definitely gonna see SageMaths speeding up and innovating ever more. It's free and it's driven by people who are passionate about it. I would bet on passionate hobbyists any day of  my life. Stephen Wolfram is one person who makes his presence felt as the face of Mathematica and his visions for the possible uses of CAM software is mind-blowing. SageMaths is driven by people from all over the world, yup everywhere. Like Linux and Android - any innovation made anywhere would be a contribution to the system as a whole.

But the thing is, for that Minors I relied on the immense simplicity of Mathematica 'cos I only had a faint idea about Sage and wasn't really willing to code. Reason being, the University study is something I find to be utterly boring and it fails to inspire me at all levels - my main interest was to be able to work with Human languages, which is another one of my Great Loves. 

Needless to say that Mathematica made it possible for me to work up the project in a short time without investing much time into getting the know-how. I barely looked at the Official Tutorial 'cos most of the functions that I needed were built in, the visualization ability is breath-taking and it even suggested the commands as I typed them in. It was like something from the future to me, as I was totally ignorant of the progresses made to facilitate coding but hey, we are all crazy and stupid when we don't know something;P

So, the projects worked out good and I grew more confident with my programming skills)

But, apart from the project work where else could I use Mathematica?

The honest answer would be, nowhere!

Once you get addicted to this Closed-Source system you would obviously appreciate the programming challenges more 'cos you know there would be a function somewhere. But what if you are supposed to solve a problem, say on some programming competition or even understand the fundamentals of how the entire thing works - you would hit the dead end.

It's not about the FOR loops or even the IF-ELSE ladder that constitutes programming. To understand how the entire thing works one needs to be able to look under the hood i.e. the Source Code. Good luck doing that with a Closed-Source system!!

Besides when one needs to work on complex problems it's often necessary to optimize the code to use the full power of the Hardware, which again, requires an access to the inner workings of the system.

Sage gives you that liberty and Python sets you free;P

As far as Raspberry Pi is concerned, the official recommendation for the Magic to really work - Python.

The Little Things keep us going.

My Haskell-Scheme-Magit project on Emacs. Isn't it just beautiful!

Though the color scheme can definitely be improved and so it shall be. It's the little things which we must learn to cherish;P

Emacs, LaTex and a Novel - A writer's paradise;P

Yeah, so these days I am trying to figure out how to handle LaTex from within Emacs. Actually I have taken up the task of porting over a badly formatted PDF, perhaps based on the photocopy of the original book, to a properly formatted PDF.

Besides this being a good learning experience for me with regards to LaTex and Emacs, I have an opportunity to explore how to structure Novels. How to really write completely non-technical documents in LaTex as well and how to format them in a way that they are legible in any ebook reader like Nook and Kindle in their respective formats.

I have a feeling that this would be an amazing learning experience for it involves A Lot of typing - about 350 pages of pure text material. Add to that the details of the structural formatting of the entire text in LaTex.

Of course, I am gonna be doing the proper versioning for the files but I am gonna do it via GitHub-windows client rather than the pure Git client. I am using the pure Git client along with the Haskell project via Magit. Keep an eye out on the To Do - 2015 post !

I will definitely provide you guys with the basic template for the way I structure the text but, as I am not the original author of the text I wouldn't be taking the liberty of uploading the full fledged LaTex source files for the book. But, honestly, I'll do my best to make the LaTex template AwEsOmE and as complete as possible.

I think, I am gonna rely on LaTex + Emacs system for most of my work in future as well. So, I guess I might as well start taking it all seriously.

Helmsman, Full Steam Ahead !!

Keeping motivated

We decide to do so many things and constantly muse about being able to charm people with our skills. After all when we see people doing it, it all seems just so simple doesn't it?

But then why is it that when we ourselves do it, it becomes so confusing and so demotivating. How do they do it, how do they maintain their motivation?

I can't speak for everyone but I am quite sure that somewhere deep down, it's the same feeling regardless of what we are engaged, regardless of the mountain we are trying to scale.

For me, it's about being able to be serious like a child at play. When I loose track of time, I know that I have had fun doing it, it's such a rush when time blends into nothingness. But yes, there are many times when it becomes monotonous and even a drag !! 

For such moments, I just think about how I would like to be able to use that skill in future. I would obviously exaggerate my ability and it's effect in future and read about people who did impossible things. When this doesn't work, I would much rather imagine I am Forest Gump and I would go through it nevertheless. Try not to overthink things.

There is just so much that I don't realize about our own abilities, something that is common to all of us. We can choose to tap into that reserve of pushing-ourselves-forward no matter what.

When we really get into it, I think all the Superficial reasons (impressing someone or sounding cool et cetera ) fade away just like that. Think of it in terms of fuel, Superficial reasons can only get you started but to sustain that amount of intensity and dedication you have to look deep with in, be brutally honest with yourself regarding your motivations. The ability to persevere involves an honest introspection. 

Two things, I find, surpass that superficial stage - doing it for oneself and the best one would be, doing it for the good of others. I don't think that we have come this far in History because of a few notable figures who seem to have done impossible things on their own but it is all because of those many who cared for others. It's not a game of selfishness, it can only take you so far, I would call it a good fuel but not the best kind.

You want to reach the stars better start thinking in terms of positive contribution to the society but, no, there is ( I hope) no need for a world-soul and all giver but rather try to find a balance between personal growth and social contribution.

I would call the ability to stay motivated as Flow, because it is just that. You can read all about self-motivation on the internet and listen to the never-ending advice from people. But I assure you ,everything will fall into place if you are doing it for the right reasons. Reasons which keep you going no-matter-what. Find your own, don't borrow and don't imitate - find your own.

I developed one such habit on it's own - writing. I love writing down what I intend to do. And I would write my thoughts, my goals on the paper and forget the thing. Later on, I realized that this is something that everyone seems to advice for self-motivation but I would say live in Future, your present will make itself right on it's own.

To Do - 2015

I am gonna keep this list as a reminder for the things that I have committed to do this year, as in putting up some meaningful work up on this blog and adding a few humble drops to the ocean of knowledge on the Internet.


* Porting over the code

    ** From Hacking Secret Ciphers to Julia

    ** From How to design computer programs to Elisp

    ** From the Little Schemer series to Elisp

* Make a project Speed_Pi to complete that Personality to Numbers and to improve skill with numbers - Python

* Cormen et al - Lua 5.3

* Master the Emacs - Orgmode and make it a habit to type only in Emacs

* Master Text Processing with Python

* Complete a project using Haskell and Magit

* MATHEMATICS

    ** Master the High School and College level Matheamtics via SageMaths.

    ** PDF's modules to keep a track of progress.

    ** Make a SageMaths code companion for College Algebra book.

* Convert a badly formatted and aged PDF to LaTex (and PDF again ) using Emacs.

* Write a program to based on Text Processing and keeping tracks of new words which I come across.

There is just one driving thought in my mind this year - To give my hundred percent and to look back at this Miracle year with pride. And it shall be so!!

A bunch of Lyrics

Hey, remember I mentioned that I am doing something with Requests and Beautiful Soup - Python Libraries.

Well, actually I was working on Anaconda scripts to download a few lyrics to songs that I couldn't stop humming. So, I thought that instead of trying to copy-paste all those lyrics - why not make a program to do it?

Never mind the lyrics and the singer btw, it's a whole different story.

In the process I learned loads about structure of web-pages and how to examine their HTML code as well. And I wanted to save it all in a UTF-8 encoded file so it took me a couple of days but I managed to do it all right. This is an Anaconda script btw.

Before I put up the code let me tell you that you might need to change the :-

> Directory

> URL's

> Tweak the code some more if you wish to download FREE lyrics from other such data in public domain.

NOTE:- Don't get naughty and don't get into trouble;P