In order to answer this question, I need to define what “freedom of expression” is and cite concrete examples of how this freedom is enjoyed. And because I am an LGBT Filipino, I will use references related to the LGBT experience of a gay man residing in the Philippines.
Let’s start with definitions. Article 19 of The Universal Declaration of Human Rights states that "Everyone has the right to freedom of opinion and expression; this right includes freedom to hold opinions without interference and to seek, receive and impart information and ideas through any media and regardless of frontiers." The American Convention on Human Rights, the European Convention for the Protection of Human Rights and Fundamental Freedoms, the African Charter on Human and Peoples' Rights, and the Philippine constitution all contain something similar.
What this means for me is that as a Filipino citizen, my freedom to express my feelings and opinions is protected. I can express my sentiments against the Roman Catholic Church when they say that homosexuality is a sin, and I am free to use the Internet in posting photos of gay men celebrating the gains of the LGBT movement. In the same manner, individuals who subscribe to the catholic faith have the freedom to express their disapproval of my sexual orientation and have the freedom to form on-line groups and exchange ideas regarding their perceived immorality of homosexuality.
Although my freedom to express my thoughts is protected, the extent to which I express my thoughts is limited by various mechanisms in different contexts. For example, under Philippine law, there are limits to what I can say about the church. In fact, a recent art exhibit that was perceived to be blasphemous was closed down because it allegedly offended the beliefs of a religious sector. In the context of social media, there are also mechanisms that attempt to limit freedom of expression where it impinges upon the rights of others. For example, as a Facebook user, I have an option to block and report users whose offensive statements target a specific set of people based on sexual orientation. I am able to use this same reporting mechanism to report Facebook groups and Facebook pages. 
If we go back to the declarations of human rights at the regional level, what's common among them is the caveat that the freedom of expression indeed has limits. The American Convention on Human Rights states that "(a) respect for the rights or reputations of others; or (b) the protection of national security, public order, or public health or morals." The European Convention for the Protection of Human Rights and Fundamental Freedoms declares that "The exercise of these freedoms, since it carries with it duties and responsibilities, may be subject to such formalities, conditions, restrictions or penalties as are prescribed by law...for the protection of the reputation or rights of others." And finally, the African Charter on Human and Peoples' Rights states that "Every individual shall have the right to express and disseminate his opinions within the law." The message that is shared by these declarations is that there are two things that limit freedom of expression; laws and the respect for the rights of others. Of course, the latter is normally protected by the former.
Now that we have defined freedom of expression, described how it is enjoyed, and identified mechanisms that limit it, I will cite one clear example of expression that definitely needs to be limited. At least based on my experience, expressions and sentiments that have a negative impact on other people may be summarized with four letters, hate. Governments have responded differently to hate speech depending on its historical context and current realities. For example, in the UK, laws prohibit speech that provokes racial hatred.  In Germany, displaying the swastika or other Nazi symbols is illegal. Similarly, there are laws that address hate speech in the US and Canada. As far as the Philippines is concerned, there is still nothing outside of the blasphemy law that protects sectors of society from hate speech. In the recent months, influential people have gone scott-free despite making statements that might have been detrimental to the security of LGBT people. The 1999 Ms. Universe runner-up , Miriam Quiambao, made statements about her definition of womanhood. This effectively sets back the gains of the transgender movement in the Philippines but there is no legal recourse to hold her accountable. Boxing champ, Manny Pacquiao allegedly declared that "Gay Men Should Be Put to Death."  Because there is no hate speech law in the Philippines, the most that concerned citizens could do was demand a formal apology. In the US, however, the response was stronger than a demand for an apology, an establishment actually banned Pacquiao from entering its premises. 
Answering the Question
In summary, the right to freedom of expression must be limited because, if left unchecked, it opens up opportunities to violate the rights of others. Exactly how do we limit it? As with any international human rights instrument, there is rarely a clear cut process to define limits. There are extraneous factors to consider such as socio-economic realities, cultures, politics, etc. But one thing remains clear; that all rights are equal and interdependent. Therefore, when my freedom allows me to impinge upon yours, then this freedom must be limited. As Justice Oliver Wendell Holmes once said, “The right to swing my fist ends where the other man's nose begins.”
 American Convention on Human Rights - http://www.cidh.org/Basicos/English/Basic3.American%20Convention.htm
 European Convention for the Protection of Human Rights and Fundamental Freedoms - http://en.wikipedia.org/wiki/European_Convention_on_Human_Rights
 African Charter on Human and Peoples' Rights - http://www.achpr.org/instruments/achpr/
 The Universal Declaration of Human Rights - http://www.un.org/en/documents/udhr/
 Philippine Constitution (ARTICLE III Bill of Rights) - www.chanrobles.com/article3.htm
 Curates vs. curators: Does art transcend 'blasphemy'? - http://www.philstar.com/Article.aspx?articleId=714316
 Facebook Help Center (Report Abuse or Policy Violations) - https://www.facebook.com/report/
 The Harm in Hate-Speech Laws - http://www.infowars.com/the-harm-in-hate-speech-laws/
 An Open Letter to Miriam Quiambao (1999 Ms. Universe Runner-Up) - http://filipinofreethinkers.org/2012/04/12/an-open-letter-to-miriam-quiambao-1999-ms-universe-runner-up/
 Update: Manny Pacquiao Did Not Say Gay Men Should Be Put to Death* - http://blogs.laweekly.com/informer/2012/05/manny_pacquiao_gays_put_to_death.php
 Manny Pacquiao Banned From The Grove Over Gay Marriage Comments (UPDATED) - http://www.huffingtonpost.com/2012/05/16/manny-pacquiao-banned-fro_n_1521441.html