Oxford Economics is one of the world’s foremost independent global advisory firms, providing reports, forecasts and analytical tools on 190 countries, 100 industrial sectors and over 2,600 cities.
It is a key adviser to corporate, financial and government decision-makers and thought leaders. Its worldwide client base comprises over 700 international organisations, including leading multinational companies and financial institutions; key government bodies and trade associations; universities, consultancies, and think tanks.
With headquarters in Oxford, England, Oxford Economics has regional centres in London, New York, and Singapore, and offices across the globe in Belfast, Chicago, Dubai, Miami, Milan, Paris, Philadelphia, San Francisco, and Washington DC. It employs over 150 people, including more than 100 professional economists, industry experts, business editors, data analysts and software developers.
The Challenge
The specific challenge of this project was to provide a long-overdue modern user experience for the company’s website which consisted of two major areas dealing with marketing and providing customers access to their subscription products. The website was expected to receive an average 1000 visitors and 4000+ page views daily.
The marketing team needed to update the marketing and pay-walled site regularly using simple and effective tools in contrast to the old website which offered little direct manipulation tooling.
From a design perspective, the new site had to be clean, elegant and simple to navigate. Oxford Economics wanted to position themselves as the top economic consultancy in the world, by conveying their global presence, a strong sense of gravitas, trustworthiness and expertise.
A big challenge was to migrate over 20 thousand subscription products to the new backend without interrupting on-going access via the old website whilst the new one was being constructed.
The old system had little documentation making it extremely difficult to decipher how its features should really work.
The OLD Oxford Economics Site
The Solution
Scoping the project was difficult due to lack of legacy documentation. We had to continue to support internal publication processes and at the very least replicate the customer access experience to the subscription areas of the site. It was imperative that subscription customers continued to have unhindered access to their subscription products – i.e. publications, reports, presentations, software modelling tools, model databases and forecast data files – and to dramatically improve the user experience. We also had to achieve a complete refresh of the marketing functions of the website. This project was seen as an important opportunity to clearly explain and communicate the value proposition and benefits of quite a complex set of product and service offerings. We, in effect, had to architect an information model that would allow all this content to be composed nicely together and support inevitable future changes. The power of the information architecture and choice of CMS became evident from the outset as more and more features and content types were added into the system.
We investigated open source CMS solutions and four candidates emerged, namely: Umbraco, AxCMS.net, BlogEngine.NET and Orchard. Orchard had a big advantage: its architecture was conceived with flexibility and usability in mind. It runs on ASP.NET MVC 4 and delivers an extensible component framework to simplify construction of MVC apps making it so much more than a CMS and allowed us to leverage our experience in building these kinds of applications.
Finally, we got in touch with Orchard’s development team who confirmed Orchard was perfectly capable of getting the job done. We had to work hard to overcome some issues with performance and document indexing. We extended the out-of-the-box product in a number of ways including customizing a tool to allow widgets to be edited in place and saved changes to be seen immediately. Some of these smart extensions we hope to return back to the open source community via the Orchard code gallery.
Architecture
Basic content management is provided by Orchard out-of-the-box. The team developed custom modules to provide authentication and authorization, to manage publications and notifications and to provide search functionality. More than 20 thousand documents were indexed using Solr.
Oxford Economics Website Architecture
The Result
After extensive research, planning and development we were able to deploy to production both the marketing website and the product subscription site. The marketing team is easily able manage content through the backend and edit in place features. Clients are able to enter the premium area and browse content according to their product subscriptions. They can make use of simple search and filtering tools to find the products they need and customise RSS/Atom feeds as they desire.
New PDF documents are produced daily by the OE economist teams in a disconnected internal workflow. Because of this, migration processes we built periodically update the CMS database and run indexing procedures. This way clients have fresh content which can be previewed and downloaded.
To quote Dr. Arvindra Sehmi, CIO of Oxford Economics, “Our web site has been beautifully engineered by SophiLabs. We followed a lean, agile development process, testing and pivoting functionality quite a number of times at the start of the project. It was important to do this because we were essentially reverse-engineering a poorly understood existing system and simultaneously trying to re-ignite our brand image to make our company, our products and our services better understood in the marketplace. At one point there was so much complexity the problem looked intractable. However, the architecture and design choices that SophiLabs made at the outset really paid off handsomely. Today I consider the website more that just a ‘site’… it is a platform on which we can do a lot of ‘cool stuff’ going forward. It will help us keep our promise to deliver new innovations that will attract new clients and lead them to love what we do. The website is the showcase of the incredible breadth of products and services that Oxford Economics has to offer. It is the most important point of contact for our customers to consume everything we have to offer. SophiLabs helped us refresh the image and brand of Oxford Economics and deliver a hugely important sales tool for the company.”
The first JS Meetup organized in Uruguay took place in the Universidad Católica del Uruguay last 6th December 2012. Despite the huge downpour we had that day, It was a successful event with 35 javascripters attendees who watched three marvelous presentations about Object Oriented Programming with Javascript, Knockout.js and Node.js.
The organizer of the Meetup was Pablo Ricco, Sophilabs and Universidad Católica were the official sponsors of the event. As in every meetup, help from any other volunteer and sponsors is highly appreciated.
The first talk about Object Oriented Programming with Javascript was given by Gabriel Chertok (a.k.a Cherta) a senior web developer with huge experience creating web applications on ruby on rails using javascript, and with a solid background on J2EE. It was very motivating and enriching. He explained the characteristics, advantages and disadvantages of using prototypes and closures on Javascript, something that every good javascript programmer should know. You can find the slides here.
The second presentation was performed by Bart Calixto, a senior ASP.NET MVC developer with large experience on javascript, specially on knockout, jQuery and sammy. He is also well known on the community because of his articles about configuration of iPhone on Uruguayan TelCos.
His presentation was focused on how to create a SPA (Single Page Application) using Knockut, it was very handy and interesting.
You can find the tutorial here, and here you can check how the application will look when it is finished.
Lastly, we were delighted by a talk lead by Eduardo Veiga about Node.js. He presented node.js, the express framework, and mongoose as the main ORM. The talk showed the pros and cons of node.js, as well as how to start a basic node.js application, the classic “hello world”. Many attendees were curious about the technology and motivated to begin to explore a new world that sounds very promising.
Everyone was invited to continue developing this community in Uruguay. Javascript is one of the technologies most widely used over the world so creating a group of javascript users that encourages continuous research and the creation of better web applications must be good.
This meetup is going to be organized on the first Thursday every two months, so the next one will be on next 7th February. See you there!
Sophilabs is proud of being one of the sponsors of the first PyCon Event in Uruguay that took place on November 10th-11th 2012 in LATU, Montevideo. Sophilabs participated as sponsor of the Conference, and some team members helped on the organization and site’s construction. Pablo Ricco helped developing the website that can be found on uy.pycon.org, Martin Prunell helped on the organization and Sebastian Sassi gave a talk about how to isolate Development Environments.
The experience was fantastic, we met friendly people in the community, we learned a lot about Python, and of course we tried to make that attendees know a little bit more about what we do at Sophilabs
The first meetings were organized by the PyUy group, the mailing list that covers different topics related with Python in Uruguay and the region. Several geeks joined the meetings and ended up organizing this Conference all together.
The event organizers did an excellent work, there were more than 250 attendees and more than 10 talks from speakers coming from Argentina, Brazil, Italy, Sweden and of course, Uruguay. Thanks to the sponsors, the event registration was free, so everyone could attend, specially the young programmers that use their OLPC (Plan Ceibal) notebooks to create applications with Python on Sugar.
We had great talks about Tryton, Raspberry PI, Artificial Intelligence, Distributed Systems, Puppet & Vagrant, Backbone.js, Tastypie, BioPython, 2D/3D rendering, among others. See the complete list here.
Both Martin, Pablo and Sebastián found this experience very valuable, we are noticing how every year more people is joining the community and the reason for that is that Python has good values to support that every programmer should encourage if they want to create beautiful software.
Next year we will be definitely promoting this event again, and Sophilabs will be probably sponsoring it one more time. Hope we can meet you their and chat about technology, trends and what we love: Python.
On 17th and 18th august 2012 we participated in one of the most exciting python community-organized events: the Django Dash.
48 hours of pure adrenaline to put all of hour skills to test in order to create a cool application. Our goal was to build something that would add value to the community and would be a starting point for us to continue working on an interesting project.
Background
Are you familiar with Try Git? You should. You can learn about it at try.github.com. It’s an interactive tutorial created by the Github team to show show in a very simple way how to work with a Git repository. It emulates a console, allowing a subset of Git commands to be executed.
Yes, your guess is correct: our idea is intended to be an evolution of Try Git. We set out to create a virtual development environment that worked as an interactive educational platform. A bit too ambitious for only 48 hours of development, isn’t it? Well, we thought of that too but we liked the idea so much that we decided the Django Dash to work on an alpha version that was good enough as a starting point.
Our score at the competition wasn’t as good as we hoped for. Anyway our coding skills were really improved, we gained a bit more of hackathon experience and most important of all we learned a lot from exchanging ideas with the community and from receiving feedback from the judges.
The Mission
The goal of Try Box is to allow the user to learn about different programming languages, libraries and frameworks by interacting with an assisted development environment.
The environment has 6 main components:
Continous Github integration
A Tree-like filesystem browsing widget to edit files
An Enhanced text editor to put in all of your magic
A bash console for executing commands as needed
A Virtual application server to test the application being built
Most important: a wizard to guide you through the steps to create an application
Milestones, Steps & Hints
The key to understand the tutorial is to know that there are a number of milestones to be reached. Each milestones is reachable by following a number of steps. The user might need hints to complete each step. A milestone is a conceptual unit that belongs to a tutorial and hints are little pieces of advice given by Try Box to help the user to go through the steps to complete the milestone.
For each step, once that it’s been completed by following all the hints, Try Box will enable the “Next” button to go to the next step. Behind scenes, Try Box back end will run a set of tests to ensure the required result of the code given by the user, and if successful it will commit the code and push it to Github.
Once the tutorial is complete, project will be available on Github along with commit history so the user can review or share code, or even use it as a starting point for a another project.
The team
Sophilab’s was represented by three of our magicians:
Educators and learners
As you can see, there are two strengths about Try Box that are very important in our opinion. It’s designed for educators. From the very beginning we took both actors into account: teachers and students. This platform must assist not only people who seek to learn, but people who want to teach about best development practices and new technologies. Everyone willing to create a tutorial will be able to do so by following a well defined set of configuration patterns.
It’s designed for those who want to learn a new technology but don’t really have the time to configure a whole new development environment. With Try Box, you won’t need to install anything on your local development box. Simply follow the tutorial and see your progress in real time.
Current Status
Github integration: 60% complete
Navigation tree: 70% complete
Editor: 80% complete
Interactive console: 30% complete
Interactive hints: 10% complete
Interactive steps: 80% complete
Virtual application server: 10%complete
Tutorial creation support: 0%
Technologies
python, django, google closure, 0mq, fabric, supervisor
Contact
If you’re interested in joining us and develop a component, please contact us at contact@sophilabs.com, we’ll be really happy to hear from you. You can also take a look at the project status at https://github.com/sophilabs/trybox
Remember, icons are needed on most of the web applications, they provide a simple way of adding elegance and usability to your site.
Nowadays, choosing the correct icon set at the beginning of a project is very important, so here I share the results of a quick research that show different options.
As you can see, I prioritize the solutions that uses @font-face because you can easily change the icon sizes, colors and apply any CSS style without loosing quality.
Entypo is a set of pictograms available as vector EPS, OpenType font and web font (coming soon). All released for free under the Creative Commons license CC BY-SA.
We couldn’t find one plain and simple example on the web so we’re posting it here in case it helps someone. Our requirements:
converting DOC and DOCX to PDF from command line, server side.
No X of course. (LibreOffice excluded).
Not just DOC (Antiword excluded).
Debian packages whenever possible (python-docx excluded).
A couple of months ago, we teamed up with Stockholm located software company Seibo Software Studios to bring to life the new version of the Scandinavian Public Library Quarterly website.
There were two major requirements: 1. Develop a the new website using WordPress 2. Improve the old website.
After meeting with Rickard Carlsson, we were able to identify most of the client’s needs and to plan a three sprint project with a sprint duration of two weeks.
The challenge here was to provide Rickard with an admin interface to create articles and group them into issues. Each issue has a cover page and covers and articles can have multiple authors.
And I almost forget about the biggest challenge: porting over 400 articles from the old site to the new one.
The first sprint was about installing WordPress, plugins, creating post types and adding custom fields. The second sprint covered the site’s look & feel, coding the WordPress widget to show the issues index and the author index. The final sprint wrapped everything up. The task of porting the content started at the very beginning and wasn’t over until the very end. As usual, we made good use of scrapy for screen-scraping the old site and dump the content into WordPress import files. The old markup was such a bad enemy that we ended editing more than 400 articles manually to make them look pretty.
At the end of the day we are very pleased with having executed the project exactly as planned (this doesn’t happen very often in the industry), with having enjoyed the project and above all with having learned a lot about scandinavian public libraries and the people who work together to get all this information available to everyone.
I’m going to share with you how to build a class that communicates with JIRA in a very easy way, and you’ll be able to learn about web scraping during the process:
#!/usr/bin/python
# -*- coding: utf-8 -*-
#
# Description: JIRA web interface methods
# Authors: sophilabs
# Copyright: Copyright (c) 2011, sophilabs.
import httplib
import re
from BeautifulSoup import BeautifulSoup
def singleton(cls):
instances = {}
def instance():
if cls not in instances:
instances[cls] = cls()
return instances[cls]
return instance
@singleton
class JiraHTTPConnection:
def init(self, domain, username, password):
print domain
self.__cookie = None
self.__headers = None
self.__connection = httplib.HTTPConnection(domain)
response = self.get("/")
data = response.read()
self.__headers = response.getheaders()
self.__cookie = [x[1] for x in self.__headers if x[0] == "set-cookie"][0]
self.__login(username, password)
def get(self, path, headers=None):
if headers is None:
headers = {"Cookie": self.__cookie}
self.__connection.request("GET", path, "", headers)
return self.__connection.getresponse()
def post(self, path, post_data, headers=None):
if not headers:
headers = {"Cookie": self.__cookie}
self.__connection.request("POST", path, post_data, headers)
return self.__connection.getresponse()
#private
def __login(self, username, password):
post_data = "os_username=%s&os_password=%s&os_destination=" % (username, password) + "%2Fsecure%2F"
headers = { "Cookie" : self.__cookie, "Content-Type": "application/x-www-form-urlencoded", "Content-Length": str(len(post_data)) }
path = '/login.jsp'
response = self.post(path, post_data, headers)
data = response.read()
class JiraWebConnection:
def __init__(self, domain, username, password):
JiraHTTPConnection().init(domain, username, password)
def projects(self):
path = "/secure/BrowseProject.jspa"
response = JiraHTTPConnection().get(path)
data = response.read()
soup = BeautifulSoup(data)
project_keys = [re.search("/browse/([a-zA-Z0-9_-]+)", unicode(x).encode("utf-8")).group(1) for x in soup("a") if "/browse/" in unicode(x).encode("utf-8")]
return project_keys
We want the JiraHTTPConnection class to be a singleton. This can be achieved using decorators . This class will be responsible for providing requests methods and taking care of cookies, and logging in to JIRA when connection is first established.
I’ve used Live HTTP Headers for Firefox to inspect the contents of the POST request during a normal login to JIRA. Then all we need is to replicate that request by creating a custom one.
So you’ll have the data containing the proper credentials and we need to add the cookie to the header as well.
Don’t forget to read the response after the request even if you’re making no use of it.
You can add this piece of code to the bottom of the file for testing:
if __name__ == "__main__":
jira = JiraWebConnection(YOUR_JIRA_URL, USERNAME, PASSWORD)
p = jira.projects()
This will return a list of project keys associated to the current user.
Customer asked us to parse a csv file. There was a lot of processing to do on the contents, but we weren’t being able to apply logic to it because lines didn’t behave as expected.
After reading the files and printing to console we got something like this:
