Posts

(DRAFT) Scaling a django project using celery and elasticsearch

Scalability has undoubtedly become an important asset not only for large software projects but also mid-level ones now. Indeed, when you have a look at developer job posts, you can see easily that the need to build scalable software is one of the top desired skills. Making a software scalable requires mostly more than one methodology and different technology stacks. At first, it may look difficult and require deep experience with those methods and stacks. And yes, scalability is a serious topic. Although it is, in this post, I will show how to make a Django application scalable. In the end, you will learn fundamental concepts and approaches. PLAN create a django project which is a little product management system. make adding product asynchronous instead of synchronous sync elasticsearch with the primary database ( SQLite ) use elasticsearch for autocomplete and real-time analysis 1. Product management app with Django Since I assume you are familiar with Python and Dj
Post a Comment
Read more

Memory organization and cache management

Image
In computer systems, caches are used to diminish the slowness of main memory which is larger and cheaper than cache memory. Caches are faster because not far from CPU and not as large as main memory. In modern computers, there may be more than one cache memory. The layers composed a memory hierarchy. The memory hierarchy is sometimes called an illusion to processors because a processor can have as large and fast a memory as it needs at the level of reasonable cost. In short, from a broader perspective it is easy to understand why memory hierarchies are needed; however, to understand how it works you need to narrow the perspective, e.g. taking account only two memory layers at one time. Key concepts are: cache miss and hit ratios. memory and cache and average access speed memory size, cache size block or cache line size set size direct mapping, fully associative mapping, set associative mapping address partitioning (tag bits, index/set bits, block bits) replacement algor
Post a Comment
Read more

Memory design and virtual memory

Image
In the class of SWE 514 Computer Systems , I am studying low-level concepts about cpu and memory design. Although I am totally abstracted from them in daily programming life, I think If you are a software developer it is good to catch at least a glimpse of how memory management works, regardless you write low-level or high-level programs. In many respect, management burdens are being carried out by operation systems; however, sometimes having knowledge about these may help you in terms of performance or security issues. I am not going to explain what the memory management is here; instead, I will share some useful resources. The article titled " Principles of virtual memory " by Carl Burch and Hendrix College seems to me that it covers most of sides of virtual memory concept very concisely. After reading it, you will understand why virtual memory exists and which parts of it are actually crucial. In the following youtube playlist, virtual memory concept is discussed as why w
Post a Comment
Read more

Injection libraries for Java, Android: Butterknife & Roboguice

Image
In this article, I will show you how to inject Android views using Roboguice and Butterknife, and how to use dependency injection using Roboguice. You can infer that Butterknife is not a dependency injection library. Why do we need them? If you start to develop programs beyond “hello world” with java or all other languages, you probably want to get rid of some repetitive, cumbersome boilerplate coding parts. Indeed, we should focus on logic, not on metaprogramming. For example, in Android development, you get view references as follows: In this code, we only want to change a textView’s text and show the labels of the two buttons. What we want is called our program’s logic. As it is obvious we have to write a lot of statements to achieve such a simple task. Now try to guess what happens if you have a lot of views, more than just two… Butterknife helps us to focus on logic With this library, it is easier for us to create view references. So you can write a shorter version
Post a Comment
Read more

Never Run Your Mongodb as Root User

We have recently had a "to many open files" problem in one of our Mongodb servers. Mongodb kept saying "Out of file descriptors. Waiting one second before trying to accept more connections." and accept no more connections. The first thing came to our mind was that the reason is too much load, our mongodb servers were no longer enough to carry it. But it was wrong. After first research it became clear that it was because of linux's user limits, so our wrong decision: running mongodb servers with the root user. In linux systems, every user has some limits which decide how much system resources they are allowed to consume. These limits prevent one devil user to drain all the system and to make other users to starve of resources. In our linux servers, root user had lower limits than mongodb's suggested limits. Even if the root user had the suggested limits, we could have had the problem any way. Because root user is responsible for other operational works, not
Post a Comment
Read more

Euclidean Distance (similarity)

Bir kümedeki elementlerin benzerliğini ölçen, benzerlik puanı üreten avantajı ve dezavantajı duruma göre değişen algoritmalar bulunmaktadır. Collaborative Intelligence kitabında Euclidean Distance hesaplaması ile film eleştirmenleri arasındaki benzerlik hesaplanmıştır. Senaryoda, kişiler çeşitli filmlere puan vermişlerdir ve bu puanlar üzerinden bir kişinin diğer insanlarla arasındaki benzerlik hesaplanmıştır.Birkaç ekleme dışında kodların tamamını kitaptaki gibi aktarıyorum. from math import sqrt def sim_euclidan_distance(prefs, person1, person2): si={} for item in prefs[person1]: if item in prefs[person2]: si[item]=1 if len(si)==0: return 0 sum_of_squares=sum([ pow( prefs[person1][item]-prefs[person2][item], 2) for item in prefs[person1] if item in prefs[person2] ]) return 1/(1+sqrt(sum_of_squares)) critics={ 'Mustafa': { "The Matrix": 5, "The Godfather": 5, "Se7en": 3, "Babel": 2, "Troy": 4, &
Post a Comment
Read more

Code is what, not how

Code is known to be written something by programmers for computers to do some tasks. Currently commanding by a language which is near to computers is not necessary for most use-cases. When we create a file on a disk, in most use cases we do not care how this process is carried out. When we fetch an URL’s content, we enjoy concise statements like ‘fetch_content(url)’, get_url_content(url)’. We are not interested in how, but what. We are migrating from the imperative paradigm to the declarative one. So that we invented high level languages such as Python, C#, Java, PHP; which are more similar to a natural language.
Post a Comment
Read more