Tongan police said the boat set off from the capital Nuku'alofa with another vessel on Monday afternoon and was due to arrive overnight in Fonoifua island, about 90 kilometres (55 miles) northeast.
One boat docked as scheduled but there was no sign of the second, prompting police to call in a naval vessel and tell other boats to be on the lookout.
A New Zealand air force P3-Orion also scoured the area for three hours without success and Maritime New Zealand (MNZ) said another Orion and a helicopter would resume the search on Wednesday.
"The suspicion is it could be the motor must've died, they must be drifting," deputy police commissioner Pelenatita Vaisuai told Radio New Zealand.
There were conflicting reports about how many children were on board the 11-metre (36-feet) boat.
Police initially reported there were six adults and five children, aged two to 12, but MNZ said it had been told there were eight men, two women and only one child.
Small private boats have been providing inter-island ferry services in Tonga since the beginning of the month, with the regular ferry out of service with a broken rudder.
Tonga suffered its worst maritime disaster in August 2009, when the ferry Princess Ashika sank, claiming 74 lives.
Tongan police said the boat set off from the capital Nuku'alofa with another vessel on Monday afternoon and was due to arrive overnight in Fonoifua island
Rescuers launched an air and sea search Tuesday for a boat missing in Pacific waters off Tonga carrying 11 people, including up to five children.
