Such clues as are available point towards Freetown, Bristol, MA -- but there were no Japhets in Freetown.  (In fact, "Japhet" as a last name appears to have been unknown in the U.S. prior to 1821.)  However, there was a Stephen Juckett, who married Hannah Piggsley in Freetown on 8 Jul 1799.  Since most of the Piggsleys changed the spelling of their name to either Pittsley or Pixley over the next few years, and since Stephen and Hannah Japhet's first child was born in 1800, this would appear to be an exact match.

A 1997 RootsWeb posting on this Juckett family of Freetown stated:

I'm looking for information on the shipwreck of a French Warship around
Cape Cod sometime prior to 1700.  The only survivors of the wreck were
Louis DeMaranville, Francis Voteau, Francis Crapo (the captain), his
brother Peter Crapo and Peter Juckett (Joquit, Jerket, Joguet, Jackett,
etc...)  The survivors are said to have washed up in Plymouth.

Peter Crapo, Peter Juckett and Louis DeMaranville all appear in the
records of Rochester and Freetown, MA, with their descendants spreading
out from there.  Juckett was married in Boston in 1714 (Rebecca Askin or
Haskin), Peter Crapo married in 1704 (Penelope White), and Louis
DeMaranville married in 1730 (Peter Crapo's daughter Susannah).  Many of
their descendants ended up in Rhode Island, New York, Connecticut and
Michigan.

Here is the line leading from Peter Joquit to Stephen Juckett:
    Peter Joquit (b.c. 1672, France) m. Rebecca Askin,  1 Jun 1714, Boston, MA 
        (marriage performed by the Rev. Dr. Cotton Mather)
    Peter Joquet (7 Feb 1714/15, Boston) moved to Freetown, Bristol, MA
        m. Miss Jones, 1734
    Peter Joquet/Juckett (c. 1735, Freetown, MA)
        m. Thankful Benson (b. 7 Aug 1740) c. 1760
        (daughter of Ebenezer Benson and Joanna Andrews)
    Stephen Juckett m. Hannah Piggsley, 8 Jul 1799, Freetown, MA

Hannah's parentage is not shown anywhere that I can find, but I'm fairly sure she must have been a granddaughter of Alexander Pigsley and Mary Westcott of Freetown, three of whose sons appear on the same 1790 census listing as Peter Juckett.  Of those three, the only one with a gap in his family where Hannah might fit is Joseph Pigsley/Pittsley.  The 1790 census suggests that he and his wife, Hannah Raymond, had five daughters at that time, but online listings only include the names of three (or at most four.) 

Alexander Pixley, who married Stephen Japhet's daughter Fannie, was the son of Joseph Pittsley's brother Benjamin.  Benjamin had died in 1796, and Joseph married the widow in 1798, thereby becoming Alexander's step-father.  Two of Joseph's sons, Alexander's older brother, and Stephen Japhet's two oldest sons all settled in Chenango County about 1815/20.  These multiple interconnections among the three families lend credence to the idea that Hannah might have been the daughter of Joseph Pittley.

The 1800 census for Freetown shows a Stephen Jucket age 16-25 with a wife age 16-25, a son under the age of 10, and a girl age 10-15.  This appears to be Stephen with his wife, their first child, and perhaps a female relative.  They appear just one page below the listing for Joseph Pigesley.

Stephen and Hannah may also have been the parents of Andrew Japhet who died in East Greene, Chenango Co. on 2 Jun 1861. 

All four children are mentioned in the wills of two of Lydia's brothers, which suggests that there were no further children born after Charles and that Lydia may have died after Charles' birth.  There are also hints that the family may have moved to Steuben County, as did Alden's brother Thomas and his family.

An Alden Japhet is mysteriously mentioned in online records as having lived in Steuben County with an unnamed wife (b.c. 1834), a son (b.c. 1857), and a daughter, Julia (b.c. 1859, Addison, Steuben, NY), who married a Mr. Harrington c. 1880.

In 1870, Nancy and Gilbert were in Lisle, Broome Co. with Harriet and Alla.  Gilbert, Frances, Lucy, and Arthur are all buried in Hope Cemetery, Newark Valley, Tioga Co.
 

Elijah Japhet (9 Oct 1815 - 6 Mar 1899) was married at Japhet Hollow to Mary G. Pendell (5 May 1816, Broome Co. - 8 Mar 1882), daughter of Oliver Pendell and Hannah Brooks.  By 1880, Elijah and Mary were was living in Berkshire, Tioga Co. with their son Elijah, while Milo and Levi were in Newark Valley, Tioga Co.  The other children appear to have remained in Broome County.

Milo and Martha are buried in Evergreen Cemetery, Berkshire.  So are their daughter Delila, their son Edward and his wife Lillian, their daughter Martha and her husband William, and Milo Ray's wife Florence.  Freddie is buried in Ketchumville Cemetery, Newark Valley.

Levi and Mary are buried in Nichols Cemetery, Waverly, as are Fred and his wife Nettie.  Charley and his wife Rosa are buried in Evergreen Cemetery, Berkshire.  Franklin and his son Ivan are buried in Hope Cemetery, Newark Valley, as is Wallace's wife Nellie.  Austin, his wife Pearl, and their four sons, are buried in Bushnell Cemetery, Newark Valley.

Sally is buried in Hope Cemetery, Newark Valley, as are Gilbert and Frances, Lucy and Arthur.

After Edna's death, Charles married Sarah Elizabeth (Lizzie) Heath (27 Jan 1866, Columbus, Franklin, OH - 26 Jun 1952, Seneca, KS), daughter of Amos Heath and Cynthia Renick, on 22 or 26 Jul 1886 in Iola, KS.  Their children were:
    Cora Elizabeth (2 Dec 1888, Iola, KS - 22 Jan 1974) m. Will Van Pelt
    Frank Charles (4 Nov 1890, Burlington, Coffey, KS - 28 Nov 1976, Denver, CO)
        m. Cecyle Susan Young, 1 May 1913
            (30 Apr 1893, MO - 1926, Scott Co., KS)
        poss. m. Corinne (3 Mar 1900 - Jun 1973, Denver, CO)
    Agnes Joy (21 Dec 1897, Iola, KS - 24 Dec 1956) m. Loyde Davis
    Mabel Lillian (or Lillian Mabel) (22 May 1900, Iola, KS - 9 Feb 1982) 
        m. Harley Edgar Wilcox, 20 Aug 1919, Hiawatha, Brown, KS
            (22 Dec 1893, Chillicothe, Livingston, MO - 28 Feb 1969)
            son of John Wilcox and Marie Epperson

William and Martha's children and grandchildren were:
    Gilbert Delevane Japhet (9 Dec 1852, Kennedy, NY - 20 Feb 1926, Iowa)
        m. Mary E. Liston (b. 1856/57, Illinois) on 24 Nov 1878 in Tama Co., IA
            Melissa (b.c. Sep 1879)
            Lillie Belle (b. 25 Oct 1884, Garwin, Tama, IA)
            poss. Walter M. (c. 1890, Cedar Rapids, Linn, IA)
    Emma J. Japhet (b. 1857/58, prob. Poland, NY)
        m. Lorenzo Finehout (b. 1855/56) on 23 Feb 1879 in Van Buren Co., MI
        son of David Finehout and Mary Jane Wright
    Amy I. Japhet (b.1857/58, Chautauqua Co., NY)
        m. Willis C. Boynton (b. 1857/58, Iowa) c. 1877 in York Co.
        (Amy may have been the same person as Isabel Japhet, daughter of 
          Martha's sister Hannah, who had married an unidentified Japhet)
    Ida Mae Japhet (prob. 24 May 1863 - 12 Jun 1947, York Co.)
        m. George Dennison Elliott on 19 Oct 1879 in Illinois
        (George b. 15 Nov 1857, d. 21 Mar 1915)
            William Isaac Elliott (30 Jan 1885, NE - 30 Sep 1955, Santa Cruz, CA)
            Elmer Earl Elliott (30 Sep 1888, York Co., NE - 6 Nov 1959, Richmond, CA)
                m. Grace Jane Rood (1897-1970) on 10 Sep 1913 in Lincoln, NE

Charles and Mary's children were:
    George H. (b. 1850, Ontario Co., NY)
        prob. married Elizabeth Hall of Oxford, NY on 21 Sep 1870
    Charles W. (b. 1852/53, Yates Co., NY)
        prob. married Emma Eliza Padgett on 4 Jan 1871
            John Everett Japhet (2 Jan 1872, Michigan - 24 Jan 1955)
            in 1880, John was living with his mother's parents in Oxford, Chenango, NY
        prob. m. Ruby Jane Osborne on 23 Feb 1874 in Ludington, Mason, MI
        prob. m. Alice Finehout on 16 Nov 1877 in Van Buren Co., MI
    Jane (8 Dec 1854, Ontario Co. - 17 Feb 1857, Ontario Co.)
    Alice J. (1856/57, Hopewell, Ontario Co. - 30 Aug 1889, Manchester, Ontario, NY)
        m. Luther Spoor (b. 1843/44) on 7 Dec 1871 in Freesoil, Mason, MI
    Etta M. (b. 1869/70)

James was married in Steuben Co., NY to Rebecca Chilson (5 Oct or 10 Nov 1840, Steuben Co. - 9 May 1905, Shelbyville, Barry, MI), daughter of Chester Chilson and Elvira Grinolds.  They moved to Michigan after the Civil War, along with Rebecca's parents and siblings.  Their children were:
    Mary Ann (1863/64, Steuben Co.) m. 16 Jun 1882, Perry Granber (Granger?)
    Thomas Henry (1867, Michigan - 7 Aug 1951, Bent Co., CO) 
        married on 9 Oct 1888 in Van Buren Co., MI to Martha Adella Cannon 
            (Della was born in 1871, d. 6 Feb 1957 in Bent Co., CO)
            daughter: Elva May (26 Jul 1889, Pine Grove, Van Buren, MI)
    Chester (23 Aug 1870, Pine Grove, Van Buren, MI)
    Harriet (1873/74, Michigan)

James Japhet's name appears (as "J.A. Jophet") in an 1891 list of New York soldiers living in Michigan.  At that time, he was in Gobleville, Bloomingdale Twp., Van Buren Co.  James and Rebecca are buried in Germond Cemetery, Wayland Twp., Allegan, MI, along with Lucinda Japhet (Nov 2, 1839 - Feb 7, 1925), who might be a second wife.