You should probably check out Newport, RI. It's a little early in the season, but you'll find several good restaurants (Black Pearl on Bannister's Wharf and Christie's on Thames Street, for example), and lots of shopping and things to do, close to the docks -- very convenient, and all within walking distance. Several good restaurants and shops on Thames Street. I highly recommend Newport.
If you're so inclined, you can take a cruise out to Block Island. You should call ahead, as I'm not sure what is open now, and it's a little out of your way, I'd imagine.
It's also a little early in the season for Martha's Vineyard, but you should be able to find an open restaurant or two in Edgartown, or perhaps Oak Bluffs. I do know that Buzzards Bay can be a little dicey this time of year.
On the tip of Cape Cod, you may also want to check out Provincetown. Several restaurants and shops should be open, and all within walking distance of the docks. Be sure to check out Commercial Street -- many restaurants and lots of quaint little shops and art galleries. The Front Street Restaurant is on Commercial Street, and it's excellent -- one of the best restaurants in P-Town. Provincetown is a very quaint and entertaining place, with lots to see and do (especially during the summer), and many places to eat. I highly recommend Provincetown.
South Portland, ME, has several marinas and lots of restaurants and shopping, all within walking distance. South Portland is a nice little New England city, and has undergone a major revitalization within the past 10-15 years. Like Newport and Provincetown, I also highly recommend South Portland, ME.
Also on the Maine coast, I've heard good things about Kennebunkport (South of Portland) and Boothbay Harbor (North of Portland), but I have no personal experience with either. You may also want to check out Belfast or Camden (also North of Portland). Quaint little towns. If you're ambitious, you may also want to check out Bar Harbor -- it's way up there in Down East Land, though.
I'm guessing that you may head across the Gulf of Maine towards Nova Scotia around the Portland, ME, area. So, Belfast, Camden, and Bar Harbor may be out of your way. No personal experience, thankfully, but I've heard that the Gulf of Maine can be a little treacherous. Be careful.
I'd try to check out Newport, RI, Provincetown, MA, and South Portland, ME. If you've got the time, Block Island and Martha's Vineyard are very cool, too; however, they might be rather dead this time of year.
I think you'll like Nova Scotia. My family (on my mother's side) is from Halifax, but I haven't been back there in about 10 years.
Sounds like a lot of fun.
Be safe, and have a great trip.