Matthew is sitting at a tax booth, and he seems to be friends with other tax collectors. A τελώνιον (telōnion) refers to a tax office, or as BDAG suggests, “toll-collection operation.” In Matthew 10:3 he is called a tax-collector (τελώνης, telōnēs).
The noun does not refer to an employee of a Roman equivalent to the IRS, but rather to a “tax-farmer.” A person bids on a contract to collect taxes in a particular area, then collect whatever they could from the people and pays the Romans what he bid and pockets the rest. “The prevailing system of tax collection afforded a collector many opportunities to exercise greed and unfairness. Hence tax collectors were particularly hated and despised as a class” (s.v. τελώνης, BDAG).
In a similar situation, Zacchaeus is a tax collector and is described by those who grumbled against Jesus as a “sinner.” Zacchaeus confessed to defrauding people and promised to make restitution (Luke 19:1-10).
Matthew may not have been a tax farmer, but rather an employee in a tax office. His role is not clear, it could be something innocent (an account, a counter), or something more blame-worth (an enforcer?). In either case, anyone working for the Romans would be suspicious, perhaps even disloyalty to Jewish nationalism.
Tax collectors are never particularly popular in any society, but Jews who worked for the Romans to collect taxes and tolls were considered to be traitors since they collected money for the occupying forces. Not all taxes are bad (people tend to like nice roads and national parks, for example). Roman taxes did pay for some level of stability in the region and provided valuable infrastructure all people enjoyed. But because of the practice of tax-farming, tax collectors “were despised as greedy, self-serving, and parasitic” (Hagner, Matthew, 238).
However, like Americans who “do not want their taxes to pay for…” whatever they do not like politically, the Jewish people assumed their money was being rounded up and sent to Rome to pay for pagan temples or other sinful activities.
In addition to the political aspects of tax collecting for the Romans, a tax collector would be in constant contact with gentiles and therefore in a state of ceremonial uncleanliness. Like the leper (8:1), the centurion (8:5-13), (perhaps) Peter’s mother-in-law (8:14-15), the demon possessed men in the cemetery (8:28-34) and the paralyzed man (9:1-8), the woman with the flow of blood (9:18-22) and perhaps even the blond and mute men (9:27-31), Matthew is in a state of uncleanliness which would keep him from entering the Temple courts or sharing fellowship with the Pharisees.
Jesus simply commands Matthew to follow him and he immediately “rose and followed” Jesus. This is the same way he called the first disciples (Matt 4:13-22). The fishermen were going about their business and Jesus called them to follow without any indication they had heard Jesus preaching prior to the call. John 1:35-42 implies the early disciples were followers of John the Baptist prior to following Jesus.
It is one thing to heal a person who is ceremonially unclean, but now Jesus is calling an unclean person to be one of his disciples. When his new disciple reaches out to his social circle, the Pharisees will become indignant and will question his disciples about this practice.
This is a clear example of Jesus’s practice of eating with people who are on the fringe of what it means to be Jewish, at least from the perspective of the Pharisees.