The families will each receive 10m rupees ($189,000; £117,013), officials said. In return they have withdrawn court cases against the marines, the BBC
Two marines - Massimiliano Latorre and Salvatore Girone - who were escorting the Italian-flagged tanker Enrica Lexie off India's south western coast in February, were charged with murder and being held in the central prison in the Keralan state capital, Trivandrum.
Italy said they fired warning shots, mistaking the fishermen for pirates.
The incident took place off the coast of the southern Indian state of Kerala in February. India said the fishermen, Valentine and Ajesh Pink, were unarmed.
The tanker Enrica Lexie was on its way from Singapore to Egypt, with a crew of 34, including 19 Indians.
Under the agreement signed on Tuesday, the compensation money will be paid to the widow and two minor children of Valentine, and to two sisters of Ajesh Pink.
Italy's consul general in Mumbai, Giampaolo Cutillo, told Reuters
the payment - 10 million rupees ($190,000) to each family - was a gesture of goodwill and not an admission of responsibility for the deaths.
reported, in return the victims' families submitted that they were not pursuing their pleadings and submissions in view of a settlement.
But the marines would still stand trial on criminal charges filed by the Kerala state government, the IANS news agency reported, according to the NY Daily
"The criminal cases against the marines filed by the Kerala government would not be affected by the settlement and would proceed in the courts," said VM Syamkumar, a lawyer representing one of the families said. "The families have no control over that."
The widow of one of the victims told Reuters
she was not happy with the compensation alone.
"The money will not compensate the loss of my husband. We want the two marines to get the maximum punishment so that such incidents will not be repeated in future," she said.